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FOREWORD 


This report was prepared by Control Data Corporation, Minneapolis, 
Minnesota 55440, for the National Aeronautics and Space Administration 
(NASA) , Earth Observation Division (EOD) , located at the Johnson Space 
Center. The work described herein was performed in partial fulfillment 
of contract number NAS9-13114 and constitutes a part of a continuing pro- 
gram at EOD to further the application of remote sensing in the management 
of our natural resources. 

As a part of this work effort Control Data performed digital image 
registration techniques on four sets of multispectral scanner (MSS) imagery 
and delivered registered data tapes for NASA use and evaluation. Major 
emphasis in regard to techniques, studies, and analyses was put on auto- 
matic processing methods. It is hoped that these investigations will in 
some measure assist NASA in a practical solution of its data base manage- 
ment objectives. 

The early phases of this work were performed by the Electro-Optics 
Department of the Research Division of Control Data. In July, 1973, this 
department became the Digital Image Systems Division, and R. L. Lillestrand 
was accordingly promoted to the position of general manager of the new 
division. 

The Control Data Project Engineer, L. 0. Bonrud, was responsible for 
the overall technical direction of the project. P. J. Henrikson, assisted 
by T. Mercier, was responsible for software development, processing, and 
analysis. Image reproduction was performed by P, A. Mazorol and J. C, Gar ley, 
and graphics and publications services were provided by F. M. Dailey, 

D. M. Olson, and A. A. Yost. 

This program was funded under the above named contract during the 
period July 1972 through August 1973. Special recognition is given to 
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R* Biz^ell and K. Hancock^ each of whom served as NASA technical monitors 
on this program, as well as to R, Hill, Chief of the Photo gramme try and 
Cartography Section, Mapping Sciences Branch at NASA JSC, for his program 
support* 

This technical report was submitted on behalf of Control Data by 
L* 0, Bonrud on May 15, 1974» The report has been reviewed and is approved* 
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ABSTRACT 


Examples of automatic digital processing demonstrate feasibility of 
registering one ERTS Multispectral Scanner (1©S) image with another obtained 
on a subsequent orbits Additionally, automatic matching, correlation, and 
registration of MSS imagery with aerial photography (multisensor correlation) 
is demonstrated* Excellent correlation is obtained with patch sizes exceeding 
16 pixels square* 

Qualities which lead to effective control point selection are distinctive 
features, good contrast, and constant feature characteristics. The size of 
the features should ideally be of such a size that correlation patch sizes in 
the range of 15 to 100 pixels square can be used* This allows sufficient 
detail to minimize noise problems associated with resolution limits of the 
imagery, and yet holds spatial distortions to negligible proportions across 
the width of a patch* 

Results of the study indicate that more than 300 degrees of freedom are 
required, typically, to register two standard ERTS-1 MSS frames covering 100 
by 100 nautical miles to an accuracy of 0.6 pixel mean radial displacement 
error. Global polynomial solutions are unsatisfactory, except over relatively 
small areas, e*g. 20 to 30 nautical miles square. A very acceptable approach 
is piecewise fitting, as with quadrilateral submatrices; however, interpolation 
can be very important dependent upon the application* 

A Control Data automatic strip processing technique (TRAK) easily demon- 
strates 600 to 1200 degrees of freedom over a quarter frame of ERTS imagery 
(and is not limited to this). Registration accuracies in the range of 0.3 
to 0.5 pixel mean radial error were confirmed by independent error analysis. 
Accuracies in the range of 0.5 to 1*4 pixel mean radial error were demonstrated 
by semi -automatic registration over small geographic areas. 


/ 
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1. INTRODUCTION 


Remote sensing of the earth's surface with electromagnetic radiation has 
shown great promise for the future management of our natural resources. This 
technology provides opportunities for detecting new resources and natural 
phenomena, as well as measuring and evaluating the extent of resources in 
such areas as agriculture, forestry, geology, hydrology, meteorology, and 
oceanography. 

The Earth Resources Technology Satellite (ERTS) program has demonstrated 
that excellent imagery can be obtained from multispectral scanners in space. 
Moreover this data is produced in very large quantities. 

To a large extent the successful utilization of such data will depend 
upon transforming this data into a known ground coordinate system or register- 
ing one image with another. It is desirable then, that these processes be 
as automatic as possible in order that the work load can be performed efficient- 
ly and economically. 

This report includes the final results and recommendations concerning 
correlation and registration of ERTS-1 MSS imagery. In general this report 
attempts to show the applicability of several geometric registration techniques 
between separate images of identical or overlapping geographic areas which ^ 

were produced at different times, under different illumination conditions, 
and in certain cases by different sensors. 


1.1 OBJECTIVES 

The motivation for this study is to provide a means of rectifying any set 
of remotely sensed imagery of a given region to a standard coordinate mapping 
system. With the latter process accomplished it will then be possible to store 
and access all such imagery by their standard grid coordinates. The resultant 
ability to compare and analyze data applying to a common geographic locus will 
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greatly facilitate such further analysis and classification as may be required 
by the Johnson Space Center ERTS program. 

Basically the process of registration of any two separate images depicting 
a common area is to choose one image of the pair as a reference and then to 
transform the other so that it will superimpose as closely as possible upon the 
chosen reference. In the simplest case, if the two images were taken under 
identical conditions, a simple translation, rotation of coordinates, and scale 
change would suffice to produce exact registration. Generally this case does 
not prevail and because of variations in sensors, viewing angle, field of view, 
optical system, and the like a simple linear transformation will not result in 
exact registry. In such cases a more complicated transformation must be applied. 

The ultimate goal to be achieved is the eventual fully automatic processing 
of ERTS imagery. To this end it is necessary to investigate both automatic and 
semi-automatic techniques of image registration. In general the methods are 
completely dependent on being able to generate an interpolative spatial transfor- 
mation based upon sets of corresponding control points in both members of image 
pairs. In one case these control points are found manually and in the other they 
are located automatically by the computer. In both cases the actual fitting 
process is carried out fully automatically by the computer. 

The immediate goal of this current study then is to show how specific 
methods of image registration developed by Control Data can be applied to 
typical examples of ERTS imagery. The results of this fitting process, with 
special regard to accuracy of registration, will then establish a means of 
judging the applicability to the overall ERTS image processing problem. Finally, 
based on these results Control Data will make recommendations concerning the 
possibility of implementing the appropriate registration algorithms on special 
purpose hardware. 


1.2 DESCRIPTION OF IMAGERY 

The image data used in this study is from the ERTS-1 multispectral scanner. 
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and in two cases the ERTS data is augmented with a photomosaic constructed from 
aerial photographs to provide a ground reference. 

The multispectral scanner repetitively scans a swath transverse to the 
satellite track on the surface of the earth using a moving mirror, and the motion 
of the satellite over the ground defines the other image coordinate. The light 
reflected from each swath is divided into four optical paths* In each path the 
light passes through an optical filter defining a given spectral band and is 
then focused upon a detector associated only with that band. The signal from 
each channel is sampled, digitized and telemetered to ground receiving stations 
using pulse code modulation techniques. The nominal spatial resolution of each 
resolution element is approximately 75 meters square. The four spectral bands 
are designated as Bands 4, 5, 6 and 7 which cover the respective intervals 0.5 
to 0.6 micrometers, 0.6 to 0.7 micrometers, 0.7 to 0.8 micrometers, and 0.8 to 
1*1 micrometers. 

Four sets of imagery (Table 1.1) corresponding to four separate geographic 
areas within the continental United States were supplied to Control Data by the 
Johnson Space Center (JSC). The designations for these four sites used in this 


TABLE 1.1. MULTISPECTRAL SCANNER IMAGERY, ERTS - 1 SATELLITE 


NAME 

NUMBER 

DATE 

Hill County, Montana 

1015-17342-MB-l 

August 7, 1972 

Trinity Bay, Texas 

1036-16192-MB-1-3Y 

August 28, 1972 


1072-16190-MB-1-3A 

October 3, 1972 


1126-16195-MB-1-3Z 

November 26, 1972 

Snook Site (Sometrville 

A07348 

August 30, 1972 

Reservoir) , Texas 

A05948 

November 28, 1972 

Imperial Valley, 

1106-17504-M4 

November 6, 1972 

California 

1124-17504-M4 

November 24, 1972 
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report are Hill County, Trinity Bay, Snook Site and Imperial Valley, which will 
be more precisely defined in the following sections. Additionally, uncontrolled 
photomosaics were supplied for Hill County, Montana and Imperial County, Cali- 
fornia. 

1.2.1 Hill County 

The Hill County study is based upon a quarter frame of ERTS-1 MSS imagery 
collected over a region containing Hill County, Montana on August 7, 1972. All 
four bands were used in the study. Associated with this set of satellite Images 
is an uncontrolled photomosaic covering the same area and constructed from aerial 
photographs. The photomosaic was digitized by Control Data to provide an equi- 
valent ground element which was about one-third of the size of an ERTS element. 

1.2.2 Trinity Bay 

The Trinity Bay imagery is that part of MSS imagery covering the upper 
Trinity Bay region in the Ga Ives ton-Hous ton, Texas area. There are three sets 
of imagery (all four bands) taken on three separate dates: August 28, 1972, 

October 3, 1972 and November 26, 1972, 

1.2.3 Snook Site 


Snook Site is a local area just north of the Sommerville Reservoir in Texas, 
Two quarter frames of MSS imagery (25 by 100 nautical miles) including the Snook 
Site near one end and Columbus, Texas near the other end were used for this 
study. The data was acquired on separate passes of the satellite and separated 
90 days in time (August 30, 1972 and November 28, 1972) . 

1.2.4 Imperial Valiev 

The final set of images is for a region of checkerboard agricultural land 
in the Imperial Valley, California. Again the data is in the form of 25 by 100 
nautical mile strips of MSS images taken on two different dates: November 6, 1972 
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and November 24, 1972. An uncontrolled photomosaic of Imperial County, California 
was used to simulate registry to map coordinates. 


1.3 SCOPE OF EFFORT 

The work effort of this project consisted of digital processing of bulk 
MSS data described in Section 1.2 and studies concerning techniques of image 
registration and error analysis. 

Digital processing included reformatting of data for use at Control Data, 
contrast enhancement, removal of noise from the image data, registration of 
imagery, and generation of 9-track tapes in MSDS format for delivery to the 
Johnson Space Center. Error analysis constitutes another aspect of digital 
processing which was used to evaluate the final results. 

The requirements for registration were to provide image -to -image registra- 
tion of bulk ISS data in two cases and image-to- ground registration in the other 
two cases. The method, whether semi-automatic or automatic, was not specified. 
Expediency in returning registered data for use by the Johnson Space Center was 
the prime consideration. Nevertheless, fully automatic registration processing 
was demonstrated in three of the four cases. 

Correlation studies investigated the applicability of automatic correlation 
of MSS data and aerial photography. These studies included the effect of 
correlation patch size, spectral band, image contrast, size and shape of image 
features, control point selection, and enhancement and resolution differences. 

Specific processing tasks are described in the following paragraphs. 

1.3.1 Hill County 

MSS image data from one pass over Hill County, Montana was first registered 
to an uncontrolled photomosaic of Hill County with semi-automatic processing. 
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A quadratic fit was made by computer on the basis of 21 control points which 
were measured manually. 

The transformed MSS data from the semi-automatic processing run was then 
registered with the photomosaic data using Control Data's automatic strip (IRAK) 
processing technique. 

These two techniques simulate registration of MSS imagery to ground coor- 
dinates. Moreover, the automatic technique demonstrated multi-sensor correla- 
tion. It is the Hill County data which vjas used for the conaprehensive studies 
of correlation which are discussed in this report. 

1.3.2 Trinity Bay 

Since a very large fraction of the image area is water, which cannot be 
correlated, and for the sake of expediency, semi-automatic processing was used 
to register three sets of MSS data with one another.* Testing of 50 control 
points showed that a linear transformation should be satisfactory. Therefore, 
a linear bi-variant polynomial transformation with six independent coefficients 
was computed on the basis of these 50 control points* 

Error analysis of the results was performed with an automatic correlation 
method (WECK) and with a polynomial error analysis using polynomials ranging 
from 3 to 12 degrees of freedom. 

1.3*3 Snook Site 


Choosing one of two quarter frames (25 by 100 nautical miles) of MSS 
imagery as the reference, the second image was registered with the reference 


*Based upon the success of strip (TRAK) processing on the other examples des- 
cribed in this report, it is anticipated that this method will work for the 
land mass surrounding Trinity Bay, also. Furthermore, a procedure has been 
developed ^ich allows TRAK to conform with assigned control point coordinates. 
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using a fully automatic strip (TEAK) processing method. 


Again error analysis was performed with correlation (WECK) and polynomial 
analysis routines . 

1.3.4 Imperial Valley 

r 

The processing of this final image set utilizes a combination of the tech- 
niques discussed before. Quarter frames of MSS imagery were first registered 
between satellite passes using the completely automatic strip correlation program 
TRAK which was mentioned previously. Then part of this data was registered 
semi-automatically to an uncontrolled photomosaic of Imperial County. For this 
latter step a linear bi-variant polynomial transformation with six degrees of 
freedom was computed on the basis of 16 control points which was measured man- 
ually. 

Error analysis was performed with correlation (WECK) and polynomial analysis 
routines, 

1.4 SUMMARY OF RESULTS 

The main objective of this work was to investigate to what extent automatic 
digital processing might be effective in registering ERTS ^BS imagery, and in 
the three cases where such techniques were employed in this work the results 
were successful. Two examples of automatic registration of MSS data obtained 
on different passes and one example of automatic registration of MSS data to a 
mosaic of aerial photographs demonstrate that registration accuracies in the 
range of 0.3 to 0.8 pixel mean radial displacement error are possible. 

These results are summarized in the following paragraphs together with 
considerations of correlation patch size, spectral qualities, control point 
selection and enhancement and resolution differences. 
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1.4.1 Semi-Automatic Versus Automatic Registration 


In the context of this discussion semi-automatic and automatic processing 
are distinguished on the basis of how the matching points are found in two 
images which are to be registered. In the semi-automatic process all matching 
point coordinates are determined manually by an operator. In the automatic 
process only a sufficient number of control points are determined manually to 
initiate the process (and minimize the search area for initial lock-on) or to 
provide adequate ground control information to match a specific ground coordin- 
ate system. The majority of matching locations are determined by automatic 
matching and correlation processes. The relative merits of these two general 
approaches will be decided, then, upon the complexity of spatial transformation 
that is required, the process performance, work load, and economic considerations. 

The results of this study indicate that more than 300 degrees of freedom 
are required, typically, to register two standard ERTS-1 frames covering 100 by 
100 nautical miles to an accuracy of about 0.6 pixel mean radial displacement 
error. By way of example, if global polynomials are used to effect the spatial 
transformation this complexity requires second order bi-variant polynomials (12 
degrees of freedom) over an area of 20 by 20 nautical miles. A linear solution 
(6 degrees of freedom) is satisfactory over an area of about 14 by 14 nautical 
miles. 


From the foregoing it follows that more than 150 match points must generally 
be determined to match ERTS-1 ICS imagery to a ground coordinate system or to 
match two frames of MSS data obtained on different passes. Thus, semi-automatic 
processing requires a considerable amount of tedious and time consuming work by 
an operator to obtain the required match points. 

By contrast the Control Data automatic strip processing technique (TRAK) has 
been demonstrated to yield excellent results with the equivalent of 600 to 1200 
degrees of freedom over a quarter frame of ERTS imagery, and the process is by no 
means limited to this range of capability. The strip process technique is accor- 
dingly capable of more than 4800 degrees of freedom over a full frame of ERTS 
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imagery. It is worth noting at this point that the strip process computes a 
new, updated warp (spatial transformation) for each line of imagery. The spa- 
tial response rate is controlled by a selectable damping distance parameter. 

Global polynomial solutions are unsatisfactory, except over relatively 
small areas, say 20 or 30 nautical miles square. A maximum of about 40 degrees 
of freedom is practical with global polynomials. At higher orders the poly- 
nomials deviate considerably between fit points. A very acceptable approach 
is piecewise fitting, as with quadrilateral submatrices formed from 4 adjacent 
match points. These techniques are applicable to either semi-automatic or 
automatic registration processes. 

The accuracy of registration was analyzed with the aid of an automatic 
correlation technique (WECK) for both semi-automatic and automatic registration 
processes. Where care was exercised in obtaining accurate control points, 
semi-automatic registration of MSS imageiT? to a photomosaic image (Imperial County) 
resulted in a mean radial displacement error of 0,5 pixel over an area less than 
20 nautical miles square. MSS data obtained from three different passes over a 
90 day time-span was registered in this manner with a resultant 0.7 pixel mean 
radial error (Trinity Bay) . With a less precise measurement procedure MSS data 
was registered to a photomosaic of Hill County to an accuracy of 1.4 pixel. 

Using the automatic strip (TRAK) process quarter frames of MSS imagery 
acquired at different times were registered to 0.28 pixel (Snook Site) and 0.58 
pixel (Imperial County) mean radial error. The transformed data obtained from 
the Hill County run described above was further improved in the TRAK run to 0.4 
pixel mean radial error. 

These results are encouraging in two respects. First, it is possible to 
register two MSS images having temporal changes with an automatic matching and 
correlation procedure. Second, it is possible to correlate ground control areas 
in a photomosaic with MSS imagery. 
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Semi-automatic processing has the advantage of less computer time and the 
disadvantage of greater clock time. Pre-processing enhancement is also desirable 
to aid the operator in feature recognition. Automatic processing has the advan- 
tages of less clock time, greater volume, and no need for pre-processing (though 
pre-processing may be desirable for other reasons). Autcsnatic processing has 
the disadvantage of more computing time, but it may have the further advantage 
of greater accuracy under given work load conditions, particularly where there 
is a lack of sharp feature detail. 


i.4.2 Image Correlation 

Cross correlation of ERTS-1 MSS data and a photomosaic of Hill County 
provides the following observations concerning the effects of patch size, 
spectral region of the sensor, and image features. 

Excellent correlation is obtained with patch sizes exceeding 16 pixels 
square in the MSS data matrix* Xt is observed that recognizable features are 
normally evident within a 16 by 16 pixel window. However, a correlation window 
4 pixels square seldom includes recognizable features. Therefore, such a small 
patch creates problems with secondary maxima and minima in the correlation 
surface. 

For the Hill County example band 5 MSS data was most similar to the photo- 
mosaic as judged by visual appearance, joint gray-scale distribution diagrams, 
and cross correlation. The mean correlation value for 21 control points varied 
from a minimum of 0.556 for band 4 data to a maximum of 0.615 for band 5 data 
(Table 2.2-1). The mean coordinate locations of maximum correlation between 
each of the image bands and the photomosaic did not differ by more than 0.2 
pixel for the 21 control points (in terms of MSS data matrix) , Likewise, band 5 
data was judged to be most reliable for matching Trinity Bay, Snook Site, and 
Imperial Valley data. 

Qualities which lead to effective control point selection are distinctive 
features, good contrast, and constant feature characteristics. The size of the 
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features should ideally be of such a size that correlation patch sizes in the 
range of 15 to 100 pixels square can be used. This allows sufficient detail 
to minimize noise problems associated with resolution limits of the imagery, 
and yet holds spatial distortions to negligible proportions across the width 
of a patch. This latter condition is important lest distortions have a signifi- 
cant effect upon the location of maximum correlation for the entire patch. 

Either man-made or natural features can be used as control points, provided 
they are not subject to significant change. One of the greatest problems here, 
of course, is seasonal variation. A solution to this problem is maintaining 
control point reference images that are typical of several seasons. 

A number of examples are analyzed to show the effect of various image 
patterns upon the correlation function. It is demonstrated that automatic 
correlation is effective even under conditions of low contrast features. 

Experimental results demonstrate that two images can be effectively matched 
and correlated even though resolution is significantly different in the two 
images. A mathematical analysis is offered to support this observation. There- 
fore, there is no need to equalize resolutions prior to correlation. Nor is 
the use of a gradient function advantageous in area correlation. 
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2. DISCUSSION OF RESULTS 


2.x ERTS REGISTRATION REQUIREMENTS 
2.1.x Techniques and OUjectlves 

In designing an image processing system there are two basic questions to 
be answered: What will work and what will work best . The second question 

is in many ways more difficult to answer than the first because it requires 
definition of user objectives and user enviroment. Given these, trade-off 
studies can be performed to establish preferred techniques and to optimize 
parameters. The question of which techniques are best for ERTS image regis- 
tration depends on the particular registration requirements and on user ob- 
jectives, particularly processing volume, time constraints and accuracy 
requirements . 

This section is concerned with the basic question: What will work? 

Only digital techniques will be considered here. Under this classification 
is, of course, a relatively wide spectrum of techniques which can be differ- 
entiated on the basis of operator intervention or interaction required. At 
the two ends of the spectrum are what will be termed "semi-automatic" and 
"fully-automatic" techniques. In both cases the computer performs a spatial 
warp "automatically." The difference between methods lies in the means for 
acquiring data on which to base the spatial warp. 

In the registration process two images are to be brought into accurate 
alignment. This may be accomplished by spatially transforming or "warping" 
either or both images. For purposes of this discussion it will be assimed 
that one image will be the reference or independent image. Only the second 
image, which will be referred to as the collateral or dependent image, will 
be spatially transformed to bring the two images into registration. Note 
incidently that the reference "image" need not be an actual image; it may 
be a map or nothing more than a set of control points. 
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The spatial warp applied by the computer to the dependent image is 
based on a mathematical fit to matching points on the reference and depen- 
dent images. For semi-automatic methods these control points are selected 
through some sort of operator intervention. The operator can match points 
visually on an interactive display, from photographic prints, etc. Point 
coordinates can be entered interactively or manually in a batch processing 
mode. Also the computer may be used to improve accuracy through automatic 
correlation techniques from manually matched points. 

In "fully- automatic” image registration the computer finds its own 
control points through automatic correlation. 

This section presents an investigation of both techniques for ERTS 
registration requirements. Two registration requirements are considered: 

• ERTS image -to -image or temporal registration. 

• Registration of ERTS imagery to ground control. 

A third registration requirement, registration of ERTS imagery to underflight 
data, was not investigated because underfllght data was unavailable. 

Requirements for image-to-image registration are different from those 
for image -to -ground registration. Therefore, different techniques may 
be required. The objectives of this investigation are twofold: to establish 

registration requirements and to compare semi- and fully-autoraatic regis- 
tration techniques based on samples of typical ERTS data. In the former 
case the emphasis is on establishing spatial warp complexity for the above 
registration requirements. In the latter case the emphasis is on comparative 
registration accuracy. 

The most difficult task in image registration is sampling the coordinate 
transformation between a given image and a reference. These samples in the 
form of coordinates of corresponding features on the reference and dependent 
Images form the basis for the spatial warp. 
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For semi-automatic registration samples are provided manually. The auto- 
matic phase in which the dependent image is spatially warped into registra- 
tion with the reference tends to be straightforward. A mathematical fit to 
the samples forms the basis for a point -by-point spatial warp of the dependent 
image. The mathematical fit can be of a number of forms, depending on the 
properties of the spatial distortion present. Given no priori knowledge of 
the form of this spatial distortion, a least squares polynomial fit provides 
a reasonable fit and is widely used [1,2,3]. Locations predicted by the fit 
will generally lie between pixels on the dependent image. Therefore some 
sort of interpolation, either nearest-neighbor or multi-point, must be used 
in conjunction with the fit. Generally the former is sufficient, but is 
again subject to the application and properties of the imagery. 

As for nearly any data processing application, the most important 
performance parameters for image registration are speed, accuracy and 
storage requirements. Storage requirements are tied to spatial warp 
complexity, particularly linear terms, e.g. scale change and rotation. 

Speed and accuracy are also tied to spatial warp complexity and to measure- 
ment accuracy in the samples provided. 

If measurement accuracy is poor, either computer aid must be enlisted 
to improve accuracy through automatic correlation or more samples must be 
provided than would otherwise be required. In the former case the computer 
actually matches control points, with the operator greatly reducing computer 
search time by basically telling it where to look. In the latter case 
registration error is reduced through least squares techniques. In either 
case the spatial complexity of the warp is an important factor. Accuracy 
in automatic correlation is directly related to spatial warp complexity. 

Also, the complexity of the mathematical fit and hence the number of samples 
required is directly related to warp complexity. 
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Processing speed is also directly related to spatial warp complexity* 

For example, the simpler the spatial warp, the simpler the techniques required 
for registration. In particular for linear transformations, the spatial 
warp can be implemented with a table-look'-up method to yield substantial 
savings in processing time [3], 

Registration accuracy and warp complexity are therefore the two most 
important considerations for semi-automatic registration. We would therefore 
like to establish warp accuracy as a function of warp complexity, and on 
the basis of this analysis recommend production processing procedures for 
ERTS imagery. 

On the other hand, feasibility of fully-automatic registration tech- 
niques for ERTS imagery need to be proven. Automatic techniques are not only 
affected by spatial warp complexity but also by the relative properties of 
reference and dependent images. In this connection it is important to 
distinguish between the two ERTS registration requirements under consideration, 
namely image- to -image and image- to -ground registration. In the following 
subsections the objectives in terras of automatic registration are twofold. 

• To establish feasibility of automatic registration for both ERTS 
image registration requirements 

• To compare registration accuracy of automatic techniques with 
semi-automatic techniques. 

Automatic registration for image- to -ground registration is the harder 
of the two ERTS registration requirements. Consequently, automatic regis- 
tration studies for this problem, presented in Section 2,2 are more basic 
than ERTS image- to- image registration studies performed in Sections 2,4 
and 2,5. In Section 2,2 we are trying to establish that a sufficient base 
exists for automatic registration. For image- to -image registration it is 
assumed that this base already exists. 
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Control Data has gained extensive experience in automatic image-to-image 
registration through its pioneering research and development in Side Looking 
Radar (SLR) change detection [4-H]. An operational system that includes a 
high-speed automatic registration capability has been built by Control Data 
for the U,S. Air Force [9]. This system is based on a unique method of 
pipeline processing known as strip processing. Additionally, Control Data 
has also developed these techniques to perform cancellation, registration, 
and change detection with aerial photography [12]. 

Strip processing was employed in this work to investigate the feasibility 
of image -to -ground correlation and image-to-image correlation. 

It is implemented in the form of a software simulator, Program TRAK. 
Program TRAK, also known as the "strip processor," is an experimental 
fortran/ COMPASS program designed for the CDC 6600 and Cyber series computers. 
It must be emphasized that this is not a production processing program 
for ERTS imagery. Rather it has been used with the intention of establishing 
feasibility and processing requirements for ERTS imagery. 


The fact that Program TRAK is a hardware simulator has operational 
significance. Successful TRAK processing of ERTS imagery implies that special 
purpose hardware can be designed for production processing of ERTS imagery. 
Such hardware can automatically process imagery one to two orders of 
magnitude faster than general purpose computers at JSC, while at the same 
time freeing the latter for image research application. 

In summary both semi-automatic and "fully" automatic registration 
techniques are employed in the following investigations. Four sets of ERTS 
imagery (Hill County, Trinity Bay, Snook Site, and Imperial Valley) were 
processed. Registration requirements, processing methods, and studies 
performed for each image set are summarized in Table 2-1, These investi- 
gations have the following objectives: 
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TABLE Z.l. SUMMARY OF ERTS REGISTRATION INVESTIGATIONS 


TEST SITE 

REGISTRATION REQUIREMENTS 

INVESTIGATIONS PERFORMED 

Hill County 

Image -to -ground 

U Polynomial Error Analysis for Semi-Automatic Registration 
2. Automatic Correlation/Registration Feasibility Study 

Trinity Bay 

Image -to -Image 

1. Polynomial Error Analysis for Semi-Automatic Registration 

2. WECK Error Analysis for Semi-Automatic Registration 

Snook Site 

Image- to- Image 

1. Polynomial Error Analysis for Automatic Registration 

2 , WECK Error Analysis for Automatic Registration 

Imperial 

Valley 

Image- to -Image and 
Image- to -Ground 

1- Polynomial Error Analysis for Automatic Image- to- Image 
Registration 

2, WECK Error Analysis for Automatic Image -to -Image 
Registration 

3* Polynomial Error Analysis for Semi-Automatic Image- to- 
Ground Registration 


• To establish feasibility of automatic registration techniques for 
both ERTS image -to -image and image- to-ground registration* 

• To establish automatic registration requirements, i*e* algorithm 
modifications and parameter settings, for ERTS image- to- image 
registration* 

• To establish spatial warp complexity requirements for both image- 
to- image and image -to-ground registration. 

• To establish accuracy of both semi- and fully- automatic registration 
techniques . 


2.1.2 Error Analysis 


Aside from establishing feasibility of automatic registration tech- 
niques, the two most important objectives of this investigation are to 
establish warp complexity requirements for both ERTS registration requirements 
and to establish accuracy specifications for both semi- and fully- automatic 
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processing. Accordingly, it is important that qualitative and quantitative 
procedures be developed for assessing registration accuracy. 

Three good methods of qualitatively assessing warp accuracy are 
color superposition, gradient superposition, and change detection. The 
latter two are used in this report to visually demonstrate the results of 
registration processing. 

In gradient superposition the gradient of the reference image is super- 
imposed on the collateral image following registration processing. The 
gradient operator emphasizes edges. Hence by thresholding the gradient of 
the reference and superimposing it on the dependent image, one can assess 
registration accuracy by assessing the degree to which edges or boundaries 
are in alignment on the two images. 

Change detection also provides a means of assessing registration 
accuracy. Spatial misregistration between reference and dependent images 
will result in black and white change pairs or "ghosting" in the difference 
image. This assumes, of course, that the two images are in registration 
radiometrically, i.e. density values for corresponding features on the two 
Images have the same distributions. If not, some sort of photoequalization 
is necessary for change detection to yield meaningful results. 

Both of these methods provide only qualitative visual results. 
Quantitative evaluation requires statistical evaluation of processing results. 
Quantitative evaluation of registration accuracy for real data is generally 
difficult because of difficulty in establishing an absolute reference. In 
laboratory simulations, of course, establishing an absolute reference is 
straightforward. Program TRAK has been tested extensively in this manner, 
and registration accuracy was found to be on the order of one- fourth pixel 
mean radial error [7.9l. But the question of interest is not how techniques 
perform in the laboratory, but how they perform on real data, i.e. "in the 
field." 
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Two types of statistical studies will be performed to evaluate spatial 
registration accuracy: polynomial error analysis and WECK analysis. In both 

cases, analysis is based on samples representing matching features on the 
reference and dependent images. In areas where there are no matching 
features, either because of temporal change or because there are simply no 
prominent features, neither method can provide a measure of warp accuracy. 

Polynomial error analysis assumes that spatial distortion between refer- 


ence and dependent 

images 

can be modelled by a 

polynomial, i.e. 
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(2.1-1) 

where (x,y),(x'y») 

are coordinates of some point 

on the reference and 


dependent image, respectively, N is the order of the polynomial, and the 
coefficients determined by a least squares fit to some set of 

control points. These control points can result from either manually 

matching features or from automatic correlation. In either case, polynomial 
error analysis can be used to compile error statistics as a function of the 
order of the warp. These statistics can then be used both to assess warp 
accuracy and to establish warp complexity. 

An important consideration in polynomial error analysis is degree of 
over-determination. Residual error at fit points will always decrease as 
the complexity, i.e. the number of unknowns or "degrees of freedom," is 
increased for a least squares polynomial fit. In the limiting case where 
the number of measurements equals the number of unknowns, the residual error 
at each fit point is identically zero. For this reason, one should, if at 

all possible, divide the set of control points into a set of fit points and 

a control set of "check" points. 
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The reason for employing check points is to measure fit error at points 
other than those employed in the fit. As fit complexity is increased, the 
fit begins to respond to "measurement noise" at the fit points. The result 
is increased error at points other than fit points. Within the set of fit 
points, fit error tends to be greatest at points farthest from the fit 
points, hence, a reasonable choice of check points is as shown in Figure 2.1-1. 


OXOXOXOXOX 

XOXOXOXOXO 

OXOXOXOXOX 

XOXOXOXOXO 

X FIT POINT 

OXOXOXOXOX 

0 CHECK POINT 

XOXOXOXOXO 

OXOXOXOXOX 

XOXOXOXOXO 

OXOXOXOXOX 

XOXOXOXOXO 

Figure 2.1-1. Typical Pattern of Fit and Check Points 
for Polynomial Warp Error Analysis 
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Separating the set of control points into sets of fit and check points 
is worthwhile only if there are enough control points to provide a reasonable 
degree of overdetermination. That is, only if the set of fit points repre- 
sents considerably more measurements than there are degrees of freedom in 
the highest order fit of interest. A good rule of thumb is that the number 
of measurements should be at least twice the number of unknowns. 

The second error analysis method makes no assumptions about the form 
of the spatial warp. This method is based on an automatic correlation 
technique which is analogous to that used in visual evaluation of the gra- 
dient superposition image. Reference and spatially warped dependent images 
are compared on a subregion basis to locate matching features. Given a 
matching feature, a hill-climbing technique is used to find the maximum 
correlation between images with an interpolation technique used to increase 
precision to a fraction of a pixel. The distance that a correlation maximum 
is displaced from the null condition is a direct measure of registration 
error for the feature in question. 

This method which has been implemented in Program WECK (Warp Error 
Check) has proved to be quite accurate in controlled tests. The program 
automatically locates matching features and measures displacements, then 
accumulates statistics over all points. A test on the correlation coefficient 
is used to establish which features are similar enough to be considered 
"common" features. Points with low correlation are rejected. 

Semi-automatic registration for this study was performed with Program 
SAW (Semi-Automatic Warp) . This program is a research (as opposed to 
production) program with extensive warp error analysis capabilities. Control 
points or "samples" were selected using two techniques. The first technique, 
applied to image-to-ground registration for Hill County, is purposely 
crude to simulate point matching with an interactive display. Points were 
matched from photographic prints of the photomosaic map and different 
bands of the ERTS MSS data with contrast enhancement. These prints were 
made from photographs taken from Control Data's Dicomed Model 36 display. 
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Measurements were made visually subject to a grid with a 25-pixel spacing. 
Measurement accuracy is approximately 2 to 3 pixels for this method. 

The second method for matching points, employed for Trinity Bay and for 
image-to-ground registration for Imperial Valley, is much more accurate. 
Points are matched from transparencies plotted on an OPTRONICS P1500 Film 
writer, and coordinates are measured with a Bendix Datagrid Digitizer. 
Accuracy is limited by a combination of image scale, 0.003 inch operator 
tolerance in directing the cursor, and sharpness of image features. 

Since the method is so straightforward, semi-automatic registration can 
almost always be made to work. However, in any high-volume production 
processing enviroment, manual intervention in the selection of control points 
creates a severe bottleneck. In a production mode automatic registration is 
therefore nearly mandatory from a time standpoint alone. Additionally, 
automatic processing is potentially more accurate because its measurement 
accuracy in matching control points is higher and because the number of 
control points that can be sampled automatically is much larger. 

2,2 HILL COUNTY 


The registration requirement for Hill County is ERTS image- to -ground 
registration, where ground control is provided by an uncontrolled photomosaic. 
The following discussion describes how ERTS-1 MSS data was registered to a 
photomosaic image with a semi-automatic process and how registration 
accuracy was subsequently improved with a fully automatic correlation and 
registration process. 

Correlation studies are discussed in some depth to assess the feasibility 
of automatic registration to ground control. Principal topics are: 
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• Data Preparation 

• Semi-Automatic Registration Processing 

• Automatic Multi-Sensor Correlation Studies 

• Correlation Patch Size 

• Spectral Considerations 

• Control Point Selection 

m Enhancement and Resolution Differences 

• Automatic Multi-Sensor Registration 

2.2,1 Data Preparation 

Data preparation includes reformatting of MSS bulk data, cosmetic 
removal of noise, and digitization of an uncontrolled photomosaic. 

MSS data was obtained in ICDS format with 8-bit encoding, 800 BPI on 
9-track tape. The data was reformatted to produce separate files for each 
of the four bands with 6-bit encoding, 556 BPI on 7-track tape. 

Since the MSS data is noisy it is desirable that the data be processed 
to detect and remove this noise. The chosen method is automatic and con- 
sists of replacing any bad data in a given line with the average of good 
data in the preceeding and following lines (Figures 2.2-1 through 2.2-4) , 

One noise problem occurs because of faulty recording or transmission 
to produce an interruption of the image data along scan lines. This defi- 
ciency is evident in the light and dark scan line streaks (Bands 4, 6 and 
7, particularly). Close inspection of the data reveals that these streaks 
are not always continuous. In such Instances only "bad" sections are 
replaced to avoid degeneration of "good" data. This problem was very 
effectively corrected to improve the appearance of the imagery. 
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Figure 2.2-1. Noise Reduction in ERTS Imagery 

ERTS-1 MSS, Band 4, Hill County, Montana 


24 



p« o 


a) Bulk Data 


b) Noise Removed 


Figure 2,2-2. Noise Reduction in ERTS Imagery 

ERTS-1 MSS, Band 5, Hill County, Montana 
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a) Bulk Data b) Noise Removed 

Figure 2.2-3. Noise Reduction in ERTS Imagery 

ERTS-1 MSS, Band 6, Hill County, Montana 
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a) Bulk Data b) Noise Removed 

Figure 2.2-4. Noise Reduction in ERTS Imagery 

ERTS-1 MSS, Band 7, Hill County, Montana 
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Another data collection problem is associated V7lth the data format grouping 
of six lines at a time. This Is particularly evident in Band 6 (Figure 2.2-3a) 
as a striated scan pattern with periodicity over six scan lines. The cosmetic 
correction technique is moderately successful here, but somewhat finer dis- 
crimination is required than that which was applied to this image. 

An uncontrolled photomosaic of Hill County was digitized in order that 
studies of correlation between the ERTS data and photomosaic data could be 
conducted. It is desirable that the encoding of the photomosaic be done at 
somewhat higher resolution than the ERTS imagery. Thus, the photomosaic 
was encoded to 64 gray levels in a matrix of 867 pixels by 1010 lines. 

In terms of ground scale one pixel in the digitized photoraosaic corres- 
ponds to 80 feet. Scale of the bulk ERTS data is approximately: 


s 


X 


100 nm (6076 ft./nm) 
2340 lines 


260 ft. /line 


s 


y 


25 nm (6076 ft./nm) 
700 pixels 


“217 ft. /pixel 


( 2 . 2 - 1 ) 


Therefore the spatial transformation required to register the ERTS MSS data 
to the digitized photomosaic represents a digital enlargement by a factor of 
approximately 3.2 in x and 2.7 in y. 

Semi-automatic registration of the MSS data to the photomosaic in pre- 
paration for correlation and automatic registration studies is described in 
the following paragraphs. 
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2.2.2 Semi-Automatic Registration 


ERTS MSS imagery was registered semi-automatically to ground control 
points over Hill County. That is, the spatial warp was performed automatically 
but it was based on manually matched control points. Each of the four bands 
of ERTS data were registered to the photomosaic with a quadratic warp based 
upon the 21 control points shown in Figure 2.2-5. 



Figure 2.2-5. Selection of Control Points on Photomosaic 
Hill County, Montana 
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To improve visual recognition of features the contrast of ERTS imagery 
was enhanced digitally- Also, to maintain maximum fidelity in reproduction 
both images were digitally enlarged to a matrix of 1734 by 2020 pixels. Grid 
spacing for the MSS imagery was 25 pixels, so measurement error was approx- 
imately 2 to 3 pixels RMS in either x or y. 

Of paramount importance in registering an ERTS image to ground control points 
are answers to two questions. How complicated is the spatial warp and how 
accurately can the MSS image be registered to ground control? Consider the 
accuracy question first. On the basis of measurement error alone mean 
radial registration error should be approximately 

- 

’'m ^ (2.2-2) 

where r_. is the mean radial error, N is the number of control points and 
is the RMS measurement error in either x or y- For measurement error estimates 
given above mean radial registration error should lie between 0.62 and 0.93 
pixels* The measured value for a quadratic fit to 21 control points is 1.36 
pixels (ERTS image scale). 

Possible reasons for this discrepancy between estimated and measured 
errors are: 

• Estimated measurement errors are low. 

• Spatial warp is not exactly quadratic. 

Results of a spatial warp error analysis performed in [l3]indicate that 
a quadratic or even a linear fit is an adequate approximation to the spatial 
warp between the photomosaic and ERTS imagery over a relatively small area 
such as Hill County. The higher than predicted mean radial error may result 
either from random local variations or from higher-order systematic variations 
not accounted for in the quadratic model. Error analysis results indicate 
that the former is more likely,, but there are really not enough control 
points to be conclusive. 
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In answer to the warp complexity quest ion, then, it appears that the 
spatial warp in ERTS imagery relative to ground control for Hill County is 
predominately linear. The ERTS data is skewed roughly 12^ relative to ground 
coordinates and its scale is different in x and y directions (Table 3*1 in [l3]) • 
At least six degrees of freedom (linear warp) are required to accurately model 
the spat ial warp. The increase in accuracy for 12 degrees of freedom (quadra- 
tic warp) is insignificant (Table 3-2 in [13])- 

2.2.3 Automatic Multi-Sensor Correlation Studies 


The foundation of automatic image registration is the correlation process. 
The first step in establishing feasibility for automatic image -to-ground 
registration is therefore to verify that automatic correlation can provide 
accurate control points for this application. That is, it must be established 
that the computer can recognize the same feature on the photomosaic reference 
and ERTS MSS Image without ambiguity. 

The correlation process is performed on a subregion basis; for this 
study square subregions are employed. Performance is therefore related to 
subregion size in addition to other considerations. These include: 

♦ Spatial distortion effects on correlation 

# Temporal change 

• Different sensor resolution 

• Different sensor spectral response. 

The effect of spatial distortion is discussed by Henrikson, Glish, and Hoyt [5]. 
Basically spatial distortion between images means that subregions on reference 
and dependent images will not be in complete alignment during correlation. 

The consequence is lower accuracy in matching control points and more "false 
locks." Effects of spatial distortion may be compensated for by "shaping" 
data prior to correlation. This is done for rotation and down- track scale 
variations with Control Data's current automatic strip processing technique. 
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For this investigation spatial distortion effects were minimized by 
performing correlation studies after semi-automatic registration of ERTS MSS 
imagery to the photomosaic reference. The remaining effects are those of 
temporal change, different sensor resolution between ERTS and the digitized 
photomosaic, and different sensor spectral response. 

In establishing feasibility for automatic ERTS image -to -ground regis- 
tration, these problems demand an answer to the basic question; 

• Can automatic correlation techniques recognize enough common features 
accurately enough to form the basis for a spatial warp of acceptable 
accuracy? 

This question of course implies a number of additional questions, namely: 

• What correlation patch size should be employed? 

• Which MSS spectral band or combination of bands yields the best 
results when correlated with the photomosaic reference? 

• Which features are generally best for correlation? 

• What effect if any does difference in resolution have? 

• Is any preprocessing necessary to improve correlation results? 

Automatic multisensor correlation and registration is obviously a very 
difficult problem. Therefore the temptation in designing a feasibility study 
is to make it too elaborate. 

The approach taken here is to make the automatic correlation feasibility 
study as simple and direct as possible. Basically the test for feasibility is 
to see if automatic techniques can correctly identify and match to at Least 
the same accuracy the 21 control points upon which the semi-automatic warp 
was based. In the course of this test answers to the above questions will 
be sought. 

For these studies correlation was performed over an NxN square subregion 
surrounding each control point. If the ERTS and photomosaic images are in 
perfect registration, the maximum of the cross correlation surface for each 
control point is at zero shift in x and y. Since registration is not perfect 
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the correlation maximum is shifted* Error analysis discloses a residual fit 
error having a mean radial value of 1*36 pixels on the ERTS scale, which 
translates to a value between 3-7 and 4.4 pixels on the digital photomosaic 
scale. Some of this error is, of course, a result of measurement error in 
the sampling processes* Accordingly, it is to be observed in the following 
multisensor correlation studies that the peak of the cross correlation 
surface is displaced radially from the null position by about 4 pixels on 
the average. 

For a given control point an NxN patch surrounding the point on the 
photomosaic is correlated with a window of the same size on a selected band 
of the spatially warped ERTS MSS imagery* The window is moved about on the 
ERTS imagery according to the pattern shown in Figure 2.2-6* 
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Figure 2*2-6. Correlation Sites for Multi-Sensor Correlation Study 
Hill County Photomosaic and ERTS-1 MSS Image 
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Shifts are made only along the x (east~west) and y (north-south) axis about 
the null position. A total of 2Ng+l evaluations of the correlation coefficient 
are made along each axis. 

Generally this pattern will not include the maximum of the cross- 
correlation surface but the position of the maximum can be estimated with 
acceptable accuracy from positions of the maxima for x and y shifts. This 
also means that absolute maximum correlation will be somewhat higher than 
local maxima for x and y shifts. The peak value given in Table 2.2-1 is 
the larger of these local maxima. 


TABLE 2.2-1. SUMMARY OF AUTOMATIC MSS TO PHOTOMOSAIC 
CORRELATION STUDY FOR HILL COUNTY 


CONTROL 

POINT 

PEAK CORRELATION VALUES 

PREDICTED DISPLACEMENT OF 

PEAK FROM NULL POSITION 


BAND 4 

BAND 5 

BAND 6 

BAND 7 

BAND 4 

BAND 5 

BAND 6 

BAND 7 

A X 

A y 

A X 

A y 

A X 

A y 

A X 

A y 


.487 

.689 

.617 

.573 

-3 

-2 

-3 

-1 

-3 

0 

-3 

-1 


.336 

.324 

.217 

.126 

0 

-2 

0 

-2 

1 

-3 

-12 

-13 


.596 

.700 

.714 

.698 

4 

-4 

4 

-3 

4 

-3 

5 

-3 


,766 

.777 

.733 

.638 

6 

-4 

6 

-5 

6 

-4 

6 

-4 

5 

,607 

.658 

.686 

.647 

3 

-5 

2 

-4 

2 

-5 

2 

-4 

6 

.602 

.638 

.648 

.638 

0 

-1 

-1 

-1 

1 

-1 

2 

-1 

7 

.455 

.508 

.597 

.601 

3 

2 

3 

2 

3 

1 

3 

1 

8 

.594 

.656 

.662 

,637 

2 

-3 

2 

-3 

2 

-3 

2 

-4 

9 

.560 

.642 

.656 

.615 

2 

0 

1 

-1 

2 

-1 

2 

-1 

10 

.609 

.677 

.696 

.709 

1 

-2 

2 

-3 

2 

-4 

2 

-5 

11 

.569 

.542 

.580 

.568 

-2 

-3 

-2 

-3 

-2 

-3 

-3 

-3 

12 

.636 

.689 

.710 

.702 

4 

-4 

4 

-4 

3 

-3 

3 

-3 

13 

.403 

.512 

.488 

.442 

4 

2 

4 

0 

5 

1 

5 

-1 

14 

.610 

.696 

.707 

.699 

-1 

-6 

-1 ! 

-6 

-1 

-6 

-1 

-6 

15 

.569 

.619 

,522 1 

.423 

2 ! 

-3 

3 

-3 

4 

-4 

4 

-6 

16 

.337 

.368 

,602 

.639 

0 

-4 

0 

-5 

-1 

-5 

-2 

-6 

17 

,627 

.770 

.649 

.578 

2 

-2 

1 ' 

-2 

1 

-3 

1 

-3 

18 

.510 

.539 

.476 

.459 

-1 

-4 

-2 

-5 

-3 

-5 

-3 

-4 

19 

.633 

.692 

.684 

.649 

2 

4 

1 

•^4 

2 

-3 

2 

-4 

20 

,541 

.604 

.594 

.556 

3 1 

-1 

2 1 

-2 

2 ! 

-2 

2 

-2 

21 

.630 

.621 

.512 

.424 

-6 

-4 

-4 I 

-4 

-4 

-4 

-4 

-4 

Mean 

.556 

.615 

.607 

.595^f^ 

1.19 1 

-2.19 

1,05 1 

-2.81 

1.24 1 

-2.86 

1.25 1 

-3.20* 

Mean Radial Displacement 

4 

.09 

4 

.04 1 

4, 

26 

4,62* 1 


^Excluding Control Point 2 
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Note that the mean radial displacement of the correlation peak, except 
for Band 7, falls within the range predicted above. Note also that peak 
correlation values (for each-west and north-south shifts only) are on the 
average about 0.6, a reasonably high correlation coefficient. Graphs of 
the correlation coefficient as a function of x and y shifts are included 
in the following discussion of correlation patch size. 

2.2.4 Correlation Patch Size 

The question of what patch size to use in correlation is a fundamental 

one. Generally the patch size must be just large enough to **capture^^ enough 

scrong features common to both images to allow an unambiguous match. If 

the correlation patch is made too large it may include areas in which 

correlation between the two images is low and the correlation coefficient 

will decrease accordingly. Also, of course, larger patches require more 

2 

computation time. Computation time is at least proportional to N . 

On the other hand, if the correlation patch is taken too small, it may 
not include enough area to uniquely define the feature of interest. There are 
also noise problems associated with "small sample statistics." Random 
local differences between images over the feature of Interest can make the 
correlation surface uneven and can lead to prominant secondary maxima. 

This is not acceptable for automatic registration. Prominent secondary 
maxima can lead to false matches, and an uneven correlation surface means 
problems for hill climbing algorithms which locate the correlation peak. 

A good rule of thumb is that the patch size be taken the same as that 
needed to visually match features uniquely on the two images. Consider control 
points 3 and 17, for example. Computer line printer grayscale dumps for these 
two control points on the photomosaic are presented in Figure 2.2-7 for patch 
sizes of N=L1, 51 and 81. For each control point the three patches relate to 
the same feature but are not precisely centered at the same point. Also each 
plot represents the negative of the image in Figure 2.2-5 with contrast enhance- 
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ment. The line printer gray-scale has 21 levels; to completely cover this range the 
minimum density over a patch is assigned to the first level, the maximum density is 
assigned the twenty-first and all Intermediate levels are scaled linearly, i.e. 


P 


out 


- Pmln) 

P - P ^ 

^max ^min 


(2.2-3) 


where Pmin ~ minimum density over patch 
Pmax “ niaximum density over patch 
Pin “ density at a point 

Pqu^ = line printer gray-level corresponding to p^^^ 


Obviously the 11 by 11 patch is too small to uniquely identify either 
feature. The 51 by 51 patch seems to be large enough to Include enough of 
the feature so that it can be readily recognized and the 81 by 81 patch 
begins to bring in additional features. For example, at control point 3 
roads become apparent in addition to the dam and reservoir. At control 
point 17 additional fields enter the picture. 


In the case of multi-sensor correlation, more features are not 
necessarily a good thing, as these additional features are not necessarily 
the same for the two sensors. For example, compare the Included features 
at control point 3 when N=51 and N=81. In the N=51 case one can generally 
recognize the reservoir and dam in all four ERTS spectral bands (Figure 
2,2-11), but the area surrounding the reservoir does not have the same 
appearance in the aerial photomosaic and ERTS MSS images (Figures 2,2-7 
and 2 .2-8, respectively) . 
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Figure 2.2-8. 


ERTS Band 5 for Control Point 3, 


81 by 81 Patch 
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Corretotion Cuefficient 



Correlntlon Co«niciBn1 



(b) North-South Shifts 


Figure 2.2-9. Cross Correlation, Control Point 3 


Effects of patch size on the cross correlation surface can be seen in 
Figures 2.2-9 and 2.2-10. These figures present cross sectional plots of 
the cross correlation surface between the photomosaic and ERTS Band 5* Plots 
are presented for east-west and north-south shifts with patch sizes defined 
by N = 11, 51 and 81. Note that the correlation surface is generally more 
uneven for the N =* 11 case with pronounced secondary maxima. Moreover, the 
location of the correlation peak is not necessarily the same for N “ 11 as 
for N = 15 or N = 81. 
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Comotation CoofKcf«»»i 



Corralation Coefftciant 



(b) North-South Shifts 


Figure 2.2-10, Cross Correlation, Control Point 17 


On the other hand for N - 51 and 81 the correlation surface tends to 
be smoother with a single well-defined peak. Location of the peak is approx- 
imately the same for N == 51 and N == 81. Note however that the peak value 
tends to be somewhat lower for the N = 81 case. This is a reflection of the 
face that the area is enough larger to include features that are unrelated 
on the two Images, 

In conclusion N = 11 represents too small a patch and N = 81 too large 
a patch, N = 51 appears to represent a reasonable compromise, although this 
value could probably be lowered somewhat with further research • Furthermore 
there is really no reason to hold patch size constant. It may be adjusted 
empirically to fit the size of a given feature. 
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In terms of ground scale and ERTS MSS resolution a 51 by 51 pixel patch 
represents an area of approximately 


S =[(51 pixels)x (80 ft./pixel)]^ = (4080 ft.)^ 0.6 miles^ (2.2-4) 

or on the order of 16 x 19 pixels in the ERTS MSS scale. 

2.2.5 Spectral Considerations 

The various imagery features are emphasized differently by each spectral 
band of detected radiation. Hence, best correlation can be anticipated when 
similar spectral sensitivities are used for two different sensors. 

The correlation studies with the Hill County data show that on the 
average Band 5 imagery is most nearly similar to the photomosaic, followed 
by Bands 6, 7 and 4 in that order. 

At control point 3 ERTS Bands 5 and 6 (Figure 2.2-11) are most similar 
to the photomosaic (Figure 2.2-7), although the feature can be recognized 
for all four bands. However, at control point 17 only Band 5 (Figure 2.2-12) 
is distinctly similar to the photomosaic (Figure 2.2-7). 

Note that in the former case where visual recognition is high for 
Bands 5, 6 and 7 the measured peak correlation values are uniformly high, while 
for Band 4 recognition is not so definite and the peak correlation is corres- 
pondingly lower (Table 2.2-1). On the other hand at control point 17, for 
which Band 5 is obviously the most nearly similar to the photomosaic, the 
peak correlation coefficient is much higher for that band than for any other. 
The joint probability distribution of gray-scale values affords another means 
of assessing similarity between images (Figures 2.2-13 and 2.2-14). In 
these charts the vertical axis represents density for the photomosaic, and the 
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Figure 2.2-11. Computer Line Printer Gray Scale Dump for 
Hill County Control Point 3, ERTS MSS. 
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Figure 2.2-12. 


Computer Line Printer Gray Scale Dump 
for Hill County, Control Point 17, ERTS 
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Figure 2.2- 14. Joint Distributions for Hill County Control Point 17 
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horizontal axis represents density for the appropriate band of ERTS , Frequency 
is displayed in gray scale form with the same contrast enhancement algorithm 
that was used above. These results were obtained by positioning patch matrices 
of 51 by 51 pixels to yield peak correlations - 

In interpreting joint distributions of this type one should look for 
distinct patterns in the shape of the distribution. If there is no geometrical 
organization the two processes under consideration are unrelated. On the 
other hand if the two processes are identical, all entries in the Joint 
distribution would Lie along a line of slope 45° passing through the origin. 

If the two processes have some degree of similarity but are not identical, 
a definite trend in the data should be in evidence. However, there will be 
some spread about the trend curve. 

For control point 3 there is obviously a considerable degree of similarity 
between the photomosaic and each band of ERTS. One means of comparing this 
degree of similarity is of course the correlation coefficient. Other important 
parameters are slope of the regression line, standard deviation perpendicular 
to the regression line, mean density for both processes and standard deviation 
for both processes. These parameters are summarized in Table 2.2-2 for both 
control points. 

For control point 3 Bands 5 and 6 are most nearly similar to the photo- 
mosaic. The correlation coefficient is high and the regression line is within 
a few degrees of 45°. Note also that contrast on the ERTS image is reasonably 
good. For Band 4 both contrast and correlation coefficient are much lower. 

Also slope of the regression line is 60°. For Band 7 correlation is nearly 
as high as for Bands 5 and 6, but contrast is about half as much. This is 
because Bands 5 and 6 are on a seven bit scale (which has been truncated at 
six bits) and Band 7 is on a six bit scale. The slope of the regression 
line must also be adjusted by a factor of two to be equivalent to that for 
Bands 5 and 6. 
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TABLE 2,2-2. SUMMARY OF STATISTICAL PARAMETERS FOR JOINT 
DISTRIBUTIONS AT PEAK CORRELATION BASED ON 
A 51 X 51 PATCH 


PARAMETER 


CONTROL POINT 3 



CONTROL POINT 17 


BAND 4 

BAND 5 

BAND 6 

BAND 7 

BAND 4 

— 
BAND 5 

BAND 6 

BAND 7 

P 


0.782 

0.790 

0.760 

0.663 

0.880 

0.710 

0.622 

^min 

0 

0 

0 

0 

0 

0 

0 

0 

p 

23.7 

23.7 

23.7 

23.7 

31.1 

31.1 

31.1 

31.1 

P 

60 

60 

60 

60 

60 

60 

60 

60 







cr 

12.6 

12.6 

12.6 

1 12.6 

14.0 

14.0 

14.0 

14.0 

P 








^min 

22 

11 

6 

0 

28 

24 

32 

14 

q 

33.7 


29.4 

12.2 

37.4 

40.0 

40.8 

18.6 

q 

max 

63 

63 

61 

25 

45 

54 

52 

23 

a 

n 

8.7 

13.2 

14.9 

7.4 

2.6 

5.4 

4.6 

1.9 

M 

0 

60^’ 

44° 

39° 

63° 

83° 

o 

CM 

76° 

85° 


5.5 

6.02 

6.23 

4.4 

2.0 

3.1 

3.1 

1.5 


p - correlation coefficient (peak value) 
p^in " minimum density for photomosaic, gray levels 
p ^ mean density for photomosaic, gray levels 
Pmax * maximum density for photomosaic, gray levels 

Cp = density standard deviation (contrast) for photomosaic, gray level 
^min “ minimum density for ERTS, gray levels 
q = mean density for ERTS, gray levels 
^max * maximum density for ERTS, gray levels 

a * density standard deviation (contrast) for ERTS, gray levels 

M 

0 = slope of regression Line, degrees 

- standard deviation perpendicular to regression line, gray levels 
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In smnmary, based on the statistical parameters of Table 2.2-2 Bands 
5 and 6 are most nearly similar to the photomosaic, and Band 7 is only slightly 
less so. Band 4 however is significantly less similar and has much lower 
contrast. 

For control point 17 Band 5 is clearly the most nearly similar to the 
photomosaic, Band 6 represents an intermediate case, and Bands 4 and 7 are 
clearly inferior. Note the very low contrast in this case. 

In conclusion, for best results Bands 5 or 6 should be used for correla- 
tion with the photomosaic. In any case, high correlation peak values are 
preferred for error reduction in the hill-climbing algorithms. Even if the 
other bands are used here, however, nearly equivalent results are expected 
since the peak correlation value is at nearly the same location in each case. 

2.2.6 Control Point Selection 


The above results provide a clue for the next question, namely how does 
one select features to be control points? From the above results it appears 
that both man made features (fields) and natural features (rivers) can be used. 

An obvious first requirement is that the feature be subject to very little long 
term temporal change, since a considerable time lapse is to be expected between 
dates at which imagery is collected for the reference data base and from ERTS. 

The two year lapse for this Hill County case can probably be considered typical. 
Clearly automatic techniques will not work in cases with pronounced short term 
temporal, e.g. seasonal, changes. 

In addition, features selected for control points must have reasonably high 
contrast on the ERTS imagery, the features must be large enough to cover a reason- 
able area and have variation in both x and y. If contrast is low the feature can 
be lost in '*noise," i.e. nonrepeatability between the reference and ERTS images. 

Generally, but not always, low contrast means lower correlation peaks 
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(a) East-West Shift (b) North-South Shift 

Figure 2.2-15. Cross Correlation, Control Point 13 


because of the noise problem- This is illustrated by control points 13 
and 16 which obviously have low contrast on the photomosaic reference 
(Figure 2-2-5)- Cross sectional plots of the correlation surface for these 
control points are presented in Figures 2.2-15 and 2.2-16. Note that the 
peak values are relatively low. 
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(a) East-West Shift (b) North-South Shift 

Figure 2.2-16. Cross Correlation, Control Point 16 


On the average low contrast can be expected to lead to low correlations. 
This can lead to problems in automatic processing because low correlation 
peaks may be rejected as secondary maxima. 

The reason for requiring variation in both x and y is to allow both the 
X and y coordinates of the control point to be accurately fixed. Hill 
County provides some good examples to illustrate this point, since most 
features (fields) run in a north-south direction. Consider, for example, 
control point 5. Features for this control points are mainly fields which 
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Corrolatron Cu«fficient Correlation Coefficient 



(a) East-West Shift (b) North-South Shift 

Figure 2, 2-17, Cross Correlation, Control Point 5 


are much longer in the north-south direction. This means that for east- 
west shifts there are distinct secondary maxima in the correlation surface 
(Figure 2. 2- 17(a)), corresponding to various parallel alignments of the 
fields. These secondary maxima are even in evidence for N = 51. 

Aside from the secondary maxima, however, the correlation peak is fairly 
sharp. This means that the longitude for control point 5 can be fixed 
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Correlation Cooflioiervt 


CorralBtion Coofficii 



(a) East-West Shift (b) North-South Shift 

Figure 2.2-18. Cross Correlation, Control Point 19 


accurately. On the other hand for north-south shifts (Figure 2.2-17 (b)) the 
correlation peak is broad. This is of course because there is very little 
variation in features in the north-south direction. It means that the 
latitude of control point 5 can not be fixed to the same accuracy as longi- 
tude. 

A similar example is control point 19 where many sharp features run in 
a north-south direction. The correlation surface therefore has a relatively 
sharp peak for east-west shifts (Figure 2.2-18(a)) . On the other hand, there 
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is very little variation in the north-south direction, and the correlation 
peak is quite broad in this direction (Figure 2.2-18(b)). 

2.2.7 Enhancement and Resolution Differences 


The foregoing results indicate that differences in resolution between 
the photomosaic reference and ERTS imagery are relatively unimportant, provided, 
of course, that features are selected large enough for ERTS resolution. A 
heuristic explanation for this result can be found from frequency considerations. 
The photoraosaic and ERTS imagery (any band) can be represented by random 
processes Pj^(x,y) and Pg(x,y), respectively. Obviously Pg(x,y) is a band- 
limited signal because of the discrete sampling interval. The cutoff frequency 
is on the order of half the sampling frequency. For purposes of discussion 
we will assume cutoff frequencies to be w in the x-direction and w in the 

X y 

y-direction. 

Then decompose Pj^(x,y) into (x,y), P^ (x,y), i.e. 

P^(x,y) = Pjj^(x,y) + (2.2-5) 

where Py[j^(x,y) represents all components of Pj^ at frequencies within the band 
limited by (w^,w^), and PM2(^>y) represents all higher frequency components. 

Even if the resolution of the two images were the same differences are 
to be expected because of temporal variation, different spectral response, etc. 
Therefore assume 

Pg(x,y) = K PM]^(x,y) + C + n(x,y) (2.2-6) 

where the constants K, C allow for difference in mean density and contrast. 

The term n(x,y) represents "noise” in the form of non-repeatabilities between 
the ERTS image and the photomosaic reference. Effects of spatial distortion 
between ERTS and the photomosaic are neglected for this discussion. 
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The correlation coefficient, assuming the same resolution would then 
take the form 


0*3^ = 


Et[pE(x,y) - PE][PM^( 2 ^>y) - 




E M, 


(2 2-7) 


where 

Pg = E[Pg(x,y)3= K Pmj + C + n 
P^^ “ E[PMj^(x,y)3 

n = E[n(x,y)] 

ag2 = E{[PE(x,y) - Pg]2 }= ^ +2K E {[ p^^(x,y) -Pj^^ ] [n(x,y) -n] } 

E{[PMj^(x,y) - } 

= E{[n(x,y) - n]^ ] (2.2-8) 

Assume that P|^j^(x,y) and the noise term n(x,y) are uncorrelated. Then 


P* = = sgn(K) (2.2-9) 

7k^ctm^ 2+ a^2 \/l +7 

! K^Mj 

As the *’noise’^ level, i.e. dissimilarity between images is increased p* 
decreases from 1 to zero (assuming K>0) . The same type of effect occurs for 
the case with different resolution. Basically the high frequency components 
for the higher resolution sensor look like "noise” to the correlation process 
and decrease the correlation coefficient somewhat. Assume that and 
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are uncorrelated and P is not correlated with the noise term n. Also define 

n 

the standard deviation of Pfi2* Then the correlation coefficient 
takes the form 


^ = E([PE(x,y) - PE][P^(x,y) - P 3 ] 

P ^ n n 


a G 
M E 


( 2 . 2 - 10 ) 


K 


sgn(K) 



There will be a reduction in the correlation coefficient, but provided that 
low frequency features dominate the higher frequency features, i.e. 




( 2 . 2 - 11 ) 


this reduction will not be too great. Obviously any attempt to correlate on 
fine structure in the photomosaic reference will be doomed to failure. 

What this means in terms of possible enhancement techniques is that there 
is no reason to equalize resolution before correlation by some sort of 
band-pass filtering on the higher resolution image. Another possible enhance- 
ment technique that is suggested from time to time is edge enhancement. 

Edges could be enhanced by some operator such as the (absolute) gradient prior 
to correlation. 

It has been our experience from processing imagery of the Houston -Baytown area 
that enhancing edges does not necessarily buy anything from a correlation standpoint. 
There are some pathological cases in which edge correlation is better than corre- 
lation with no enhancement, but generally if imagery is dissimilar edge enhancement 
enhances the dissimilarities and correlation is less successful. This is even 
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more apparent in this case. Results shown in Figures 2.2-19 and 2.2-20 for 
control points 3 and 17 respectively, are typical. Boundaries have distinctly 
different shapes in the ERTS and photomosaic images, and applying the gradient 
only enhances these differences. Note from the joint distributions that there 
appears to be no relation at all between the two gradient images. 

2.2.8 Automatic Multi-Sensor Registration 

The results described above indicate that automatic correlation is 
indeed feasible for image -to -ground correlation. The successful application 
of a continuous automatic registration process to Hill County data is 
described in the following paragraphs. 

Generally the computer can match control points as well as a human 
operator or better really, since the computer doesn’t need any form of 
contrast enhancement. Further, successful correlation has been demonstrated 
between ERTS MSS imagery and a photomosaic with a correlation patch 
measuring only 16 by 19 pixels in terms of ERTS resolution. Finally enhance- 
ment of either or both images to match resolution or enhance edges is not 
necessary. 

The one form of pre-processing that is necessary prior to correlation 
is ’’shaping” ERTS data to remove rotation and scaling effects. This can be 
done implicitely as data is loaded into memory and is done automatically in 
strip processing, assuming of course that the strip processor has been 
properly initialized. At this stage strip processing software (Program TRAK) 
is initialized by entering control points. In a production processing enviro- 
raent this initialization phase would have to be automated, probably based on 
ERTS satellite housekeeping data. An interesting example of automatic 
registration of ERTS MSS imagery to a photomosaic was prepared with the Hill 
County data (Figure 2.2-21). This was done with the TRAK process using the digi 
tal photomosaic data as the reference and the semi-automatically warped MSS data 
as the collateral image. In this manner MSS images for each of the four 


56 



JOINT DISTRIBUTION 
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for Hill County, Control Point 3. 
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Figure 2.2-20. 


Automatic Correlation on Gradient 
Images for Hill County, Control Point 17 
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Figure 2.2-21* Automatic Registration of ERTS-MSS Image to Photomosaic 
Hill County, Montana 


59 


bands over the Hill County Area were prepared in registration with the photo- 
mosaic image* 

In the previous discussion of semi-automatic processing the mean regis- 
tration error was shown to be about 4 pixels in the digital photomosaic format 
or 1.4 pixels in the bulk MSS format. This is visually demonstrated by 
superimposing a gradient image of the photomosaic on the MSS image as shovTn 
for Band 5 in Figures 2.2-22 and 2.2-23 (selected portions of Hill County). 



Figure 2.2-22. Automatic Registration of ERTS MSS Image to 
Photomosaic (Gradient Mosaic Superimposed on 
MSS Image - Hill County, Montana) 


reproducibility of the 
ORIGINAL PAGE IS POOR, 
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Figure 2.2-23. Automatic Registration of ERTS MSS Image to 
Photomosaic (Gradient Mosaic Superimposed on 
MSS Image - Hill County, Montana) 


Automatic IRAK processing very effectively improved the overall registra- 
tion to within 1 pixel error in the photomosaic or about 0.4 pixel in the bulk 
MSS format. This is visually demonstrated with two zoom views of the gradient 
photomosaic superimposed on the registered MSS image (Figures 2.2-22 sod 2.2-23). 


In conclusion, automatic ERTS image -to -ground registration is feasible 
and TRAK processing appears to be capable of meeting this registration re- 
quirement. Automatic registration is discussed further in subsequent sections 
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as it was applied to the registration of MSS data acquired on different passes 
of the satellite. These results also demonstrate that IRAK processing can 
yield accuracy superior to that obtainable liy semi-automatic methods. 

2,3 TRINITY BAY 

The registration requirement imposed for ERTS-1 MSS imagery obtained over 
Trinity Bay, Texas, was simply temporal image -to-image registration. That is, 
images obtained on August 28, October 3, and November 26, 1972 were to be 
registered to one another without any correction to a ground coordinate system. 

The August 28 image was chosen as the reference (Figure 2.3-1), and the 
October 3 and November 26 images (Figures 2,3-2 and 2.3-3, respectively) were 
registered to the reference. This small area of interest is dominated by the 
bay which contains non correlating detail. ' For this reason, and also because 
the relative distortions are not complex over such a small area, a semi- 
automatic warp process was chosen. Spatial transformation was effected with 
linear polynomials and was based upon 50 manually selected control points in 
each case. 

Control points were measured much more accurately for this imagery than 
in the case of the image-to-ground registration for Hill County, Montana. 
Transparent images of Band 5 MSS data were first prepared upon photographic 
film with an Optronics P-1500 Photowrite System. Then matching coordinates 
were measured with a Bendix Datagrid digitizer to an accuracy of approximately 
1 pixel. 

Registration accuracy is demonstrated visually by superimposing a threshold 
gradient image of the reference image upon each of the other two registered 
images (Figures 2,3-4 and 2.3-5), In this way the outline of predominant 
features in the reference are shown in white upon the dependent registered 
images- These results, as well as color composite analysis performed at the 
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Figure 2.3-1. Reference Image for Trinity Bay Registration 
Processing, ERTS MSS Band 5, August 28, 1972 
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Figure 2.3-2. Trinity Bay ERTS MSS Band 5, October 3, 1972 
after Registration to August 28, 1972. 
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Figure 2.3-3. Trinity Bay ERTS MSS Band 5, November 26, 1972 
after Registration to August 28, 1972. 
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Figure 2.3-4. Gradient of Reference Image Superimposed upon 
Registered Dependent Image Trinity Bay, 

ERTS-1 MSS, Band 5, October 3, 1972. 
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Figure 2.3-5. Gradient of Reference Image Superimposed upon 
Registered Dependent Image Trinity Bay, 

ERTS-1 MSS, Band 5, November 26, 1972. 


reproducibility of the 
original page is poor 


Johnson Space Center, indicate accurate registration. 

These visual tests however are not precise enough to provide a quantita- 
tive measure of registration accuracy. Therefore, the registered data was 
analyzed with the polynomial warp error analysis and automatic correlation 
(WECK) analysis procedures described in Section 2,1,2, Polynomial warp error 
analysis (Table 2,3-1) is based on the approximately 50 control points used 
for the semi-automatic warp, WECK error analysis (Table 2,3-2) is based upon 
automatic correlation at 33 locations in one pair and at 41 locations in the other. 

Results of the two analyses are in close agreement with registration 
accuracies on the order of 0,7 to 0,8 pixel RMS for a linear warp. The spa- 
tial transformations consist predominantly of translation. Rotation is on the 
order of 0,1 degree or less, and scale differences between images are on the 
order of 1% or less. Registration accuracy for a three-degree-of- freedom 
spatial warp is one to two pixels mean radial error. By introducing more 
degrees of freedom registration error can be further reduced, but little is 
to be gained beyond the linear terms. 

In Table 2,3-1 the three-degree-of-freedora spatial transformation (warp) 
consists of a translation plus rotation, i,e, 

x’ = aQ + X cos 0 + y sin 0 

y’ = bp + X sin 9 + y cos 0 (2,3-1) 

where (x,y), (x’,y’) represent coordinates of a given point on the reference 
and collateral image, respectively. The x direction is taken parallel to the 
orbiting direction, and y is taken parallel to the scan line direction. 

The four-degree-of -freedom warp consists of a translation, rotation and 
uniform scaling, i.e. 
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TABLE 2.3-1: POLYNOMIAL WARP ERROR ANALYSIS OF SEMI-AUTOMATIC 

REGISTRATION OF ERTS-1 MSS IMAGERY, TRINITY BAY 

Registration of 10-3-72 to 8-28-72 

No. of Fit Points = 50 Rotation between images = 0.08° 

Scale Compression in x*= 0.44% Scale Expansion in 0.20% 


DEGREES OF 


FIT ERROR STATISTICS (PIXELS) 

FREEDOM 

•^x 


r 

<^r 

3 

0.82 

0.82 

1.02 

0.54 

4 

0.74 

0.87 

1. 00 

0.53 

5 

0.63 

0.75 

0.84 

0.48 

6 

(linear) 

0.49 

0.62 

0.70 

0.36 

12 

(quadratic) 

0.47 

0.61 

0.69 

0.34 


Registration of 11-26-72 to 8-28-72 

No. of Fit Points = 47 Rotation between images = 0.15° 

Scale Compression in x*= 1.16% Scale Expansion in y*= 1.24% 


DEGREES OF 


FIT ERROR STATISTICS (PIXELS) 

FREEDOM 


°y 

r 

Of 

3 

1.82 

2.00 

2.44 

1.11 

4 

1.75 

2.06 

2.43 

1.11 

5 

0.60 

0.94 

0.92 

0.62 

6 

(linear) 

0.53 

0.90 

00 

o 

0.59 

12 

(quadratic) 

0.52 

0.89 

0.84 

0.59 


*X is taken parallel to the flight direction for the reference (August 28) image. 

*Y is taken parallel to scan lines, i.e. perpendicular to the flight direction 
for the reference image. 
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TABLE 2.3-2. SUMMARY OF WECK ERROR ANALYSIS FOR TRINITY BAY 
FOR SEMI-AUTOMATIC REGISTRATION 


Correlation Patch Size = 51 x 51 
Correlation Rejection Threshold = 0.45 


REGISTRATION 

NUMBER OF 


FIT ERROR 

STATISTICS 

CASE 

SAMPLES 

Sc 


r 


10- 3-72 







to 

8-28-72 

41 

0.62 

0.71 


0.69 

0.66 

11-26-72 







to 

8-28-72 

33 

0.80 

0.69 


0.73 

0.83 


x' = a^ + x(M cos 9) + y(M sin 0) 

y' = t>o “ (2.3-2) 

and the f ive-degree-of-freedora warp consists of a translation, rotation and 
differential scaling in x and y, i.e. 

x' = Bq + x(Mjj cos 6) + y(My sin 0) 

y’ = Bq - x(Mj^ cos 0) + y(My sin 0) (2.3-3) 

Statistical parameters in the two tables are defined as follows: 
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Ojj = standard deviation in the direction of flight. 

CTy * standard deviation along a scan line, 
r * mean radial error. 

aj. = standard deviation in radial error, (2.3-4) 

2.4 SNOOK SITE 

An excellent example of automatic image -to -image registration is discussed 
in this section. Quarter frames of ERTS-1 MSS imagery containing significant 
temporal changes in the form of clouds and vegetation changes were automatically 
registered to a mean radial displacement error of 0.28 pixels (Figure 2.4-1). 
Each image represents an area measuring 25 100 nautical miles. 

Snook Site is specifically a small area adjacent to the Somerville 
Reservoir north of Columbus, Texas. It can be found in the central part 
of each image in Figure 2.4-1. The specific requirement was to obtain image- 
to-image registration for this local test site. However, this was considered 
an ideal test case for automatic correlation techniques for three reasons: 

• A complete 25 by 100 nautical mile strip of data was supplied for both 
images . 

• There are significant temporal differences between the two images over 
the 90 days between ERTS passes. 

• One of the images contains partial cloud cover. 

In other words there is sufficient data on which to experiment, and the data 
represents "typical" ERTS imagery with typical problems such as changing 
vegetation and cloud cover. 

2.4.1 Process Results 


The computer program used to process this data is Program TRAK, also 
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Reference 
November 28, 1972 


Registered Dependent 
August 30, 1972 


Figure 2.4-1, Automatic Registration, Snook Site, Texas 
ERTS-1 MSS, Band 5 


72 



known as the "strip*' or "pipeline” processor. This program is a simulator 
for the ARRES automatic change detection system built by Control Data for 
the U.S. Air Force [2], The objective in processing ERTS data with this 
program is to demonstrate that high-speed high-accuracy fully automatic image 
registration is feasible for ERTS. 

Program TRAK is not a production processing program. Moreover because 
it was designed for side -looking-radar imagery, many algorithms and processing 
parameters need modification for ERTS imagery. With such modifications 
Program TRAK can form the basis for production processing software. Moreover, 
because TRAK is a hardware simulator, continued development of this technique 
can ultimately lead to special-purpose production processing hardware. 

The Band 5 data of November 28, 1972 is chosen as the reference because 
it is cloud free (Figure 2.4-1), A number of control points were measured 
at the north end (beginning of recorded data) to provide accurate initializa- 
tion of the automatic process. Processing with the TRAK program then proceeded 
automatically and continuously without interruption from the north end to the 
south end of the image strip. A gray scale threshold was imposed upon the data 
used for correlation to minimize the effects of clouds upon the correlation 
and matching process. In this manner the Band 5 data of August 30, 1972 
(dependent image) was accurately registered to the reference image (Figure 2.4-1). 

A tonal difference image was generated by subtracting pixel gray-scale 
values in the dependent image from corresponding values in the reference 
(Figure 2.4-2), The difference image provides a means of assessing real 
changes and registration accuracy. Misregistration is detected by black and 
white pairing (ghosting) of given image features. Note that no such ghosting 
is apparent. It is also important to note the preponderance of temporal 
changes in the river bottom near the Somerville Reservoir. 

In the manner discussed in Section 2.1.2 a composite gradient superposi- 
tion image was generated by superimposing the gradient of the reference image 
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Emphasized Tonal Gradient of Reference 

Difference Image Superimposed upon Registered 

Dependent Image 




Figure 2.4-2. Tonal Changes and Automatic Registration 
Accuracy, Snook Site, Texas 
ERTS-1 MSS, Band 5, November 28, 1972 
(Reference) and August 30, 1972 (Dependent) 
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(November 28, 1972) upon the registered dependent image data of August 30, 

1972. (Figure 2.4-2) ♦ Registration accuracy can be assessed by comparing the 
white gradient boundaries from the reference image with feature detail in the 
background image. 

Figure 2.4-2 certainly indicates good registration over the entire 25 
by 100 nautical mile area. Yet, registration accuracy is somewhat difficult 
to assess because of the scale and of degeneration in the half-tone printing 
process for this report. For these reasons enlargements of two specific areas 
of interest are provided for analysis in greater detail. The two areas con- 
sidered are Snook Site near the Somerville Reservoir and an area near Columbus, 
Texas. Reference, registered dependent, tonal difference, and gradient super- 
position images for Snook Site are shown in Figures 2.4-3 through 2.4-6, 
respectively. Some "popcorn" clouds and their shadows are present in the depen- 
dent image (Figure 2.4-4). Note also the differences in fields near the river 
running from the upper left to the right central area. These changes took 
place between August 30 and November 28, and not only are many of the fields 
different in density, but they also differ in shape. Furthermore, the average 
tone value is significantly darker on the August 30 image. 

Differences in fields are readily apparent in the tonal difference image 
(Figure 2.4-5) j clouds are also in evidence. From the standpoint of registra- 
tion accuracy, it appears that there is possibly some ghosting near a landing 
strip on the left and along the river. Registration appears to be reasonably 
accurate, however. 

From the gradient superposition image (Figure 2.4-6) it can be seen that 
many boundaries are not the same on the two images because of temporal change. 
Also there is a certain amount of noise amplification in the gradient process. 
Generally, though, wherever common, identical features can be identified on 
the two Images, their boundaries appear to be in proper alignment. Along the 
river there appears to be some slight misregistration in places. Note in this 
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Figure 2,A~3, Reference Image, Snook Site 

ERTS-1 MSS, Band 5, November 28, 1972 


REPRODUCIBILITY OP THE 
ORIGINAL PAGE IS POOR 
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Figure 2.4-4. 


Automatic Registration of Dependent Image, Snook Site 
ERTS-1 MSS, Band 5, August 30, 1972 
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Figure 2.4-5. 


Emphasized Tonal Difference Image, Snook Site 
ERTS-1 MSS, Band 5, November 28, 1972 
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Figure 2.4-6. Gradient of Reference Superimposed 

Upon Registered Dependent Image, Snook Site 
ERTS-1 MSS, Band 5, November 28, 1972 
(Reference) and August 30, 1972 (Dependent) 



connection that the threshold gradient image for a feature such as a river 
will consist of two lines, one on either side of the feature. In such a case 
the feature (river) should fall between the lines. 

Compared with the entire 25 by 100 nautical mile area, Snook Site should 
demonstrate the poorest registration accuracy because it has the fewest common 
features to correlate on. There are two reasons for this: the preponderance 
of temporal change in this area and clouds and cloud shadows on the dependent 
image. In light of these considerations, registration accuracy in this area 
is quite good. 

Similar results are presented for the Columbus area in Figures 2.4-7 
through 2.4-10. Contrast is generally greater in the reference image 
(Figure 2.4-7) as compared with the registered dependent image (Figure 2.4-8). 
Note also that the Colorado River appears dark on the reference (November 28) 
image and light on the dependent (August 30) image. The river therefore gives 
rise to a relatively strong change event as can be seen in the tonal difference 
image (Figure 2.4-9). 

Note that there is very little ghosting in the difference Image, an indi- 
cation of good registration accuracy. This is further substantiated by results 
displayed in the gradient superposition image of Figure 2.4-10. Again, features 
that can be identified appear to be in alignment. Note particularly highways, 
and again note the double line on the gradient image, 

2.4,2 Registration Error Analysis 

Inspection of the displacement vectors required to register the dependent 
image (August 30) upon the reference image (November 28) reveals variations 
of 2 pixels parallel to the orbital track and 2 to 21 pixels perpendicular to 
the track (Figure 2.4-11), Thus the raw data is misregistered in the direction 
perpendicular to the orbital track by amounts varying from about 0.1 to 1 
nautical mile, which is significant in many earth resources measurements. 
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Figure 2.4-7. 


Reference Image, Columbus, Texas 
ERTS-1 MSS, Band 5, November 28, 1972 
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Figure 2.4-8. 


Automatic Registration of Dependent Image, 
ERTS-1 MSS, Band 5, August 30, 1972 


Columbus , 


Texas 


REPRODUCTBILriY OF THE 
ORIGINAL PAGE IS POOR_ 
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Figure 2.A-9. Emphasized Tonal Difference Image, Columbus, Texas 
ERTS-1 MSS, Band 5, November 28, 1972 
and August 30, 1972 
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Figure 2.4-10. Gradient of Reference Superimposed Upon 

Registered Dependent Image, Columbus, Texas 
ERTS-1 MSS, Band 5, November 28, 1972 
(Reference) and August 30, 1972 (Dependent) 


84 




TRANSVERSE DISTANCE, CELLS 


700 


600 U 


500 [. 


400 h 


300 K 


200 h- 


!00 K 



200 400 600 600 1000 1200 1400 1600 

DISTANCE ALONG ORBITAL TRACK, CELLS 

WARP VECTORS 

Start of vector represents a honKontal 
displacement ol *25 cells and a vertical displacement of 0. 






























y 

M’ 




y 

/ 

y 




y 

/ 

/ 




/ 

/ 



x' 

/ 

/ 

/ 

/ 



/ 

/ 

/ 

/ 

/ 
1 

/ 

L 

/ 

1 

/ 

/ 

t 

1 

1800 

2000 


2200 



SCALE OF VECTOR COMPONENTS 


along orbital track 


SCALE IN IMAGE SPACE 


to cells 


along transverse scan l. 


along orbital track 
along transverse scan ^ 


10 nautical miles 

I 


10 nautical miles 


Figure 2.A-11. Vector Displacements Between ERTS-1 MSS Images 

at Snook Site, Texas, August 30 and November 28, 1972 




In addition, there is an additional translation o£ the data files along the 
orbital track amounting to 25 pixels which was automatically compensated in 
the registration process. Registration accuracy over the entire 25 by 100 
nautical mile strip (700 pixels by 2340 lines) was measured with the correla- 
tion error (WECK) analysis method following registration of Band 5 data in 
the TRAK process. The mean radial displacement error is 0.28 pixel (Table 
2.4-1) . 


TABLE 2.4-1: SNOOK SITE: CORRELATION ERROR (WECK) ANALYSIS 

OF AUTOMATIC (TRAK) REGISTRATION PROCESS 
OVER 25 BY 100 NAUTICAL MILE AREA 


ERTS-1 MSS, Band 5, Image Data 
August 30 and November 28, 1972 


Correlation Patch Size = 51 x 51 Pixels 
Correlation Rejection Threshold = 0.45 
Total Number of Samples =315 


DESCRIPTION 

SYMBOL 

PIXELS 

Mean Registration error in direction of flight 

X 

0.007 

Standard Deviation in direction of flight 

a 

X 

0.19 

Mean Registration error parallel to scan lines 

y 

-0.05 

Standard Deviation parallel to scan lines 

a 

y 

0.30 

Mean radial error 

r* 

0.28 

Standard Deviation in radial error 

a 

r 

0.21 


An important question related to registration accuracy is warp complexity. 
Just how complex does the spatial warp have to be to yield an accuracy on the 
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order o£ % pixel? It would appear from inspection of the vector displacement 
diagram that a linear warp may generally be sufficient over areas of about 
400 X 400 pixels as in the Trinity Bay example* Over the complete frame, 
however, the warp is of higher order* 

A quantitative evaluation of the warp complexity was made with a poly- 
nomial warp error analysis performed on control points matched by Program TRAK. 
The set of control points was divided into sets of "fit” and"check" points 
(Figure 2.1-1), Polynomial fits of order N, where N varies from one to four, 
were applied to the fit points, and residual error statistics were computed 
over both the fit points and interspersed check points (Table 2.4-2). 

TABLE 2.4-2: SNOOK SITE: POLYNOMIAL ERROR ANALYSIS 

OF AUTOMATIC (TRAK) REGISTRATION PROCESS 


Based upon control points matched by Program TRAK 
over a 25 by 100 nautical mile area. 


Number of fit points - 88 
Number of check points - 95 
Total number of points » 183 


ORDER OF 
FIT 




ERROR STATISTICS 

(PIXELS) 



— 1 

FIT POINTS 

CHECK POINTS 

ALL POINTS 1 

a 

X 

a 

y 

r 

a 

^ 

a 

y 

r 


a 

y 

r 

1 

IQ 

0.75 

1.16 


1.25 

1.20 




2 

mm 

0.71 

0,70 

mm 

1.30 

0.89 




3 

mm 

o 

o^ 

0.66 

0.46 

1.35 

o^ 

o 




4 


0.60 

0.58 

0.51 

1.31 

0.88 





^■1 










<j - standard deviation in fit residual, direction of flight 
X 

Gy = Standard deviation in fit residual, parallel to scan lines 
r mean radial error 
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A test against the correlation coefficient for each control point was 
made to ensure that points employed in the error analysis represent good 
matches of corresponding features on the two images* The results are based 
on a total of 88 fit points and 95 check points distributed over the complete 
25 by 100 nautical mile strip* 

It is apparent that the spatial warp is globally second order (quadratic). 
That is, the best global fit is of the form 


2 2 

x' = + &2y + + ®5y 

y' = + bj^x + b 2 Y + b^xy + b^x^ + b^y^ (2.4-1) 

where (x,y), (x*,y‘) represent coordinates of the reference and dependent 
images, respectively. As in the previous examples, x is measured parallel 
to the flight direction, and y Is measured parallel to the scan lines. With 
12 unknown coefficients equation 2.4-1 represents 12 degrees of freedom. 

These fit coefficients for the 25 by 100 nautical mile area are given in 
Table 2.4-3. 

Clearly six degrees of freedom are not sufficient, since mean radial error 
over all fit points is decreased 32% by Increasing the number of degrees of 

freedom from six to twleve, i.e. by increasing fit order from one to two. 

On the other hand, fits higher than second order lead to no significant 
further decrease in mean radial error. 

It is interesting to compare these results with those for the WECK error 
analysis and also with results of error analysis for the Trinity Bay Imagery. 
From the coefficients in Table 2.4-3 and the vector displacement diagram it 
can be seen that locally, i.e. over an area of 400 to 500 pixels on a side as 
for Trinity Bay, the spatial warp is predominantly linear. Note also that 
a quadratic warp over the 25 by 100 nautical mile strip (700 pixels by 2340 


88 



TABLE 2.4-3. QUADRATIC FIT COEFFICIENTS FOR SPATIAL TRANSFORMATION 
BETWEEN ERTS-1 MSS IMAGES AT SNOOK SITE 
(25 BY 100 NAUTICAL MILE STRIPS) 


Reference: November 28, 1972 

Dependent: August 30, 1972 


POWER 
OF X 

POWER 
OF y 

COEFFICIENTS 

COEFFICIENTS 

FOR DIRECTION 

FOR SCAN LINE 

OF FLIGHT 

DIRECTION 

0 

0 

-25.32 

-22.39 

1 

0 

0.9945 

0,0054 

0 

1 

-0.0044 

1.016 

1 

1 

6.75 X 10"^ 

-8.50 X 10‘® 

2 

0 

2.48 xlO"^ 

-7.50 xlO"^ 

0 

2 

2.20 xlO"^ 

1.16x lO"^ 


lines) at Snook Site and a linear warp over the smaller Trinity Bay area 
(400 pixels by 500 lines) yield similar mean radial errors of about 0.8 
plxe 1 . 

A comparison of the polynomial error analysis and the WECK error analysis 
of Snook Site processing reveals a significantly lower mean radial error for 
the TEAK spatial warp. What this means is that the spatial warp between ERTS 
frames may deviate locally from a low-order global fit. Program TRAK, which 
updates the spatial warp on a line by line basis can respond to these local 
deviations to decrease the mean registration radial error from on the order 
of 3/4 pixel to on the order of 1/4 pixel. 
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2.5 IMPERIAL VALLEY 


Two objectives for registration of ERTS-1 MSS imagery of Imperial Valley 
were: 

• Automatic image -to -image registration (MSS imagery only). 

• Semi-automatic registration to ground control points, simulating 
transformation to Universal Transverse Mercator (UTM) grid projection. 

As the following discussion and examples will shown, these objectives were 
well attained. 

2.5.1 Automatic Image -To-lmage Registration (MSS) 

The procedure used in processing this image pair was to first perform 
image -to- image registration with automatic techniques (Program IRAK), then to 
register both images to ground control semi-autoraatically (Program SAW) . The 
images are compared with the preparation of difference images, and registration 
accuracy is evaluated with gradient superposition images and statistical error 
analysis. 

Automatic registration processing was performed on MSS Band 5 data for a 
25 by 100 nautical mile area in Imperial Valley. The image obtained on 
November 6, 1972 was chosen as the reference (Figure 2.5-1). The dependent 
image (November 24, 1972) after registration to the reference is also shown 
in Figure 2.4-1. Within the limits of scale factor and half-tone reproduction 
for this report a gradient superposition image shows good registration over 
the entire strip (Figure 2,5-2). The white boundaries are the threshold 
gradient of the reference image . 

A tonal difference image was created without equalization of the gray- 
scale distributions in the reference and registered dependent image (Figure 
2.5-2). This was done to preserve the radiometric integrity (within the limits 
of the interpolation process) for possible signature analysis. As might be 
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Reference 
November 6, 1972 



Registered Dependent 
November 24, 1972 


Figure 2.5-1. Automatic Registration, Imperial Valley, California 
ERTS-1 MSS, Band 5 
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Emphasized Tonal 
Difference Image 


Gradient of Reference 
Superimposed Upon Registered 
Dependent Image 




Figure 2.5-2. Tonal Changes and Automatic Registration Accuracy, 
Imperial Valley 

ERTS-1 MSS, Band 5, November 6, 1972 (Reference) 
and November 24, 1972 (Dependent) 
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expected by viewing the original images the differences are great, particularly 
in the fields of the central area and also in the southwest corner of the strip. 

The extent of these tonal differences are further analyzed with the aid of 
enhanced three-level difference images. A simple nonlinear enhancement algo- 
rithm is employed in which all differences greater than twice the threshold 
setting are set to the maximum (63 on a six bit scale). All differences less 
than twice the negative of the threshold are set to the minimum (zero) , and 
all in between values are set to an intermediate shade of gray ,(32) , Thus as 
the threshold is raised only the strongest changes remain as black or white 
areas on a gray background. Thresholds of 8 and 10 are shown here (Figure 
2.5-3). 


It appears from the tonal difference image that there is a definite photo- 
equalization problem for these two images. There is little doubt in this case 
that density distributions are considerably different for the two dates, as is 
further demonstrated by histograms of grayscale distributions for the two 
original images (Figure 2.5-4). Yet, feature detail is remarkably similar 
throughout the image, and dissimilarities are not as extensive as for the 
Snook Site imagery. This can be expected because of a shorter interval (18 
days) between passes over Imperial Valley as opposed to 90 days for Snook Site. 
Furthermore, the Imperial Valley imagery has very well defined features of 
good contrast. 

The foregoing observations are In complete agreement with the obtained 
result that correlations between the Imperial Valley images were significantly 
higher, typically in the range of 0.7 to 0.8, as opposed to 0.6 to 0.7 for 
Snook Site. 

The scale factor of Figure 2.5-2 does not provide a good opportunity to 
accurately assess registration by visual inspection. Since the area of pri- 
mary interest is Imperial County (Figure 2,5-5), a 500 by 500 pixel section 
of the MSS imagery including this area is digitally enlarged for closer in- 
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Figure 2.5-3. Three Level Difference Image, Imperial Valley 

ERTS-1 MSS, Band 5, November 6, 1972 (Reference) 
and November 2A, 1972 (Dependent) 
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Frequency X Frequency X 




D1805 

Figure 2 •5-4. Gray Scale Distributions for Imperial Valley 
EKTS-1 MSS Imagery, Band 5 
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Prepafed by the Mapping Sciences Branch. £aah Observation Division, Seienco 
& Applications Directorate. Manned Spaeecrah Center Houston. Texas 


UNCONTROLLED MOSAIC 



ICXX5 Meter UTM GrO. Zone II 


Figure 2.5-5. Uncontrolled Photomosaic 

Imperial County, California 
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spection (Figures 2.5-6 and 2.5-7), Superposition of the gradient reference 
image upon the registered dependent image reveals that registration is generally 
within one pixel over this selected area (Figure 2.5-8). Note that this data 
is extracted from the entire 25 by 100 nautical mile strip which was previously 
registered with the automatic strip (IRAK) process. 


The mean radial displacement error was found to be 0.58 pixel for IRAK 
processing of the quarter frame of MSS imagery in image -to -image registration 
(Table 2,5-1). This result is the accumulation of statistics over 373 loca- 
tions at which independent correlations were made on the registered image data 


TABLE 2.5-1; IMPERIAL VALLEY: CORRELATION ERROR (WECK) ANALYSIS 

OF AUTOMATIC (TRAK) REGISTRATION PROCESS 
OVER 25 BY 100 NAUTICAL MILE AREA 

ERTS-1 MSS, Band 5, Image Data 
November 6 and November 24, 1972 


Correlation Patch Size = 51 x 51 Pixels 
Correlation Rejection Threshold = 0.60 
Total Number of Samples =373 


ERROR STATISTICS 


DESCRIPTION 

SYMBOL 

PIXELS 

Mean Registration error in direction of flight 

— 

X 

-0.04 

Standard Deviation, direction of flight 

'"x 

0.52 

Mean Registration error parallel to scan lines 

y 

-0.08 

Standard Deviation parallel to scan lines 

O' 

y 

0.75 

Mean radial error 

r“ 

0.58 

Standard Deviation in radial error 

a 

r 

0.70 
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Figure 2.5-6. 



Reference Image, Imperial County 
ERTS-1 MSS, Band 5, November 6, 1972 
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Figure 2.5-7. Registered Dependent Image, Imperial County 
ERTS-1 MSS, Band 5, November 24, 1972 
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Figure 2.5-8. Superposition of Gradient Reference 
Upon Registered Dependent Image 
for Imperial County, California Area 
ERTS-1 MSS Band 5, November 6, 1972 
(Reference) and November 24, 1972 (Dependent) 
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with a correlation patch size of 51 by 51 pixels. 


Thus, the registration of Imperial Valley MSS data is not quite as good 
as that of Snook Site where the mean radial error was only 0,28 pixel. Further- 
more the spatial warp between images appears to be somewhat more complex, as 
evidenced by the vector displacement diagram (Figure 2.5-9*) and a polynomial 
warp error analysis (Table 2,5-2). A third or fourth order fit is needed to 
reduce mean radial registration error to the same level as a second order fit 
provides for Snook Site (Table 2.4-2) . 

It is interesting to speculate why the Imperial Valley imagery should have 
a higher registration error, even though temporal change is less and correla- 
tions are higher. From the vector displacement diagram it appears that the 
process may have experienced occasional local difficulties in correlation, 
e.g. in the beginning (north end) and in the south west. Otherwise there were 
indeed complex local distortions between the two images. 

Actual elevation induced distortions in the mountainous regions of either 
image may be in the range of 1 to 5 pixels. However, the apparent flight 
paths differ by about 0.6 degrees in heading and only about 2 nautical miles 
in translation perpendicular to the flight path. The heading deviation could 
account for a maximum of 0.1 pixel relative displacement perpendicular to the 
scan lines in the regions of the hi^ mountain ridges. If the lateral trans- 
lation is valid, this could account for a maximum of 0.2 pixel relative 
displacement in the scan line direction in the mountain peaks, also. Instabil- 
ity of the satellite orientation would add further to these local distortions. 
Thus, it is possible that the higher residual error encountered here did arise 
in part from terrain induced distortions which were not compensated by the 
strip correlation process. 


*In the vector displacement diagram of Figure 2,5-9 each vector has been biased 
by (-215,-44) to remove the translational offset between images. Also the 
vectors have been enlarged by a factor of 10 in the horizontal (flight) direc- 
tion and a factor of 5 in the vertical (scan line) direction. 
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Figure 2,5-9. Vector Displacement Diagram for Spatial Distortions 
between ERTS Images for Imperial Valley, 

November 6 to 24, 1972 


TABLE 2.5-2. IMPERIAL VALLEY: POLYNOMIAL ERROR ANALYSIS 

OF AUTOMATIC (TRAK) REGISTRATION PROCESS 


Based upon control points matched by Program TRAK 
over a 25 by 100 nautical mile area. 


Number of fit points = 74 
Number of check points = 71 

Total number of points = 145 


ORDER 
OF FIT 

ERROR STATISTICS (PIXELS) 

FIT POINTS 

CHECK POINTS 

ALL POINTS 


C7 

cr 

Y 

a 

mm 

r 

a 

O' 

r 


X 

y 





X 

y 


1 

0.96 

1.04 

1.10 

1,07 

1.22 

1.18 

1. 01 

1,13 

1.14 

2 

0,80 

0,87 

0.91 

0,93 

0,98 

1,00 

0.86 

0.92 

0.96 

3 

0,75 

0,68 

0.78 

0.94 

0.67 

0.88 

0,84 

0.67 

0,83 

4 

0.66 

0,61 

0.72 

0.94 

0,54 

0-86 

0.81 

0.57 

0.79 


In addition, local variations in features cannot be ignored as possible 
causes in hi^er residual error. This raises the question of accuracies 
relating to man-made features as opposed to natural features. Also the noise 
line problem every sixth line seems to be more severe for the Imperial Valley 
ERTS imagery. More discussion of warp complexity follows in the discussion 
of registration to ground control over the Imperial County area. 
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2.5.2 Semi-Automatic Registration to Ground Control 


As stated at the outset, one of the objectives is to simulate spatial 
transformation of MSS imagery into registration with a standard grid projection, 
such as the UTM grid. This was done with a linear semi-automatic warp based 
upon 16 control points measured from an uncontrolled photomosaic (Figure 2. '5-5) 
and Band 5 data of the reference MSS image (Figure 2.5-6). The resulting 
transformation was applied to all four spectral bands of imagery for November 6 
and November 24, 1972. (Figures 2.5-10 and 2.5-11, respectively). Properties 
of this fit, including the fit coefficients, are described in Table 2.5-3. The 
mean radial error is 0.77 pixel. 

TABLE 2.5-3: IMPERIAL COUNTY: LINEAR POLYNOMIAL FIT 

FOR SEMI-AUTOMATIC REGISTRATION OF 
ERK-1 MSS IMAGERY TO GROUND CONTROL POINTS 

Order of Fit: First (linear) - 6 Degrees of Freedom 

Number of Control Points: 16 

Mean Radial Error in Fit (Based on 16 Control Points) “0.77 Pixel 


POWER 
OF X 

POWER 
OF y 

FIT COEFFICIENTS * 

X TERM 

y TERM 

0 

0 

222.6 

68.1 

1 

0 

0.974 

0.330 

0 

‘ i 

-0.170 

1.341 


^Coordinate .Convention 

Photomosaic Reference; x = east-west direction 

y = north-south direction 
ERTS MSS Reference; x = flight direction 

y = scan line direction 
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Band 6 Band 7 


Figure 2.5-10. Semi-Automatic Registration 

to Photomosaic, Imperial County 
ERTS-1 MSS, Band 5, November 6, 1972 
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Band 4 


Band 5 



Band 6 Band 7 


Figure 2.5-11. Semi-Automatic Registration 

to Photomosaic, Imperial County 
ERTS-1 MSS, Band 5, November 24, 1972 
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The semi-automatic warp was scaled to provide a "one-to-one" digital trans- 
formation (same digital scale) in the flight direction. In the perpendicular 
direction (along a scan line) it was then necessary to compress the digital 
scale by approximately 31 percent. 

Results of a polynomial registration error analysis based on the 16 con- 
trol points are summarized in Table 2.5-4. Up to third order (cubic) polynomial 
fits are considered. Registration error drops dramatically as the number of 


TABLE 2.5-4. IMPERIAL COUNTY; POLYNOMIAL ERROR ANALYSIS 
OF SEMI-AUTOMATIC REGISTRATION PROCESS 


Based on 16 ground control points obtained 
from an uncontrolled photomosaic. 


DEGREES OF 

REGISTRATION ERROR STATISTICS 

(PIXELS) 

FREEDOM 

a 

X 

CT 

y 

r 

a 

r 

4 

9.12 

12.19 

13.93 

4.74 

5 

5.50 

7.10 

8.26 

2.80 

6 

0.51 

0.73 

0.77 

0.40 

(linear) 





12 

0.36 

0.55 

0.58 

0.26 

(quadratic) 





20 

(cubic) 

0.34 

0.41 

0.45 

0.25 


a = standard deviation in direction of flight 
Oy = Standard deviation parallel to scan lines 
7 “ mean radial error 


CJ^ = standard deviation in radial error 
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degrees of freedom is increased from three to six. Beyond this point regis- 
tration error decreases much more slowly with increasing fit complexity. 
Registration error cannot be accurately assessed for higher order fits, say 
cubic and above, because of the limited number of control points available. 

The data in Table 2,5-A indicates that a linear fit over a 500 by 500 pixel 
area should be adequate (mean radial error =0.77 pixel), and that some addi- 
tional improvement is possible with a quadratic fit (mean radial error = 

0.58 pixel) . A linear warp of ERTS IKS imagery requires at least six degrees 
of freedom to account for differences in x and y scale factors, as well as a 
skew of about 12 degrees relative to a ground coordinate grid. 

Though the four examples of MSS registration discussed in this work provide 
a limited sampling, it would appear that these examples provide a beginning 
point in requirements projections. The results indicate that it is perhaps 
not uncommon to find that about 12 degrees of freedom are required for MSS 
to ground registration over a 500 by 500 pixel area (380 square nautical miles) 
to reduce the mean radial registration error to 0.5 pixel or less. If so, a 
minimum of about 80 degrees of freedom are required for a quarter frame, and 
more than 300 are required for a full frame of standard ERTS Imagery, 
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3. RECOMMENDATIONS POR AUCTIENTING THE GENERAL IMAGE PROCESSING 

CAPABILITY AT NASA JOHNSON SPACE CENTER 

Image processing can be classed in two main categories: production pro- 

cessing and research processing. Computer requirements are quite different 
for these two categories, and an image processing computer facility must be 
capable of meeting requirements for both categories. The key concept in the 
design of an image processing facility is therefore versatility. Control Data 
feels that the most cost-effective means of attaining such versatility is to 
combine the computing power of a large scale general-purpose computer with an 
array of high-speed special-purpose image processing computers. 

Consider computer requirements for research processing. Research pro- 
cessing is generally performed on a low-volume basis with a high degree of 
interaction required between researcher and computer. Each computer program 
must be, tailored to a particular application, of which there are generally a 
wide variety at any image processing facility. Algorithms tend to be rather 
sophisticated and in a state of constant revision. In image research each 
researcher is attempting to extract information from ERTS data for his special- 
ized application. This may require several iterations. First one technique 
is tried and results analyzed, then another, then perhaps some parameters are 
changed, etc. 

Key computer requirements for research processing are therefore computer 
availability and computer flexibility. Each researcher must have ready access 
to the computer — either through an interactive display capability or through 
short turn-around time in a batch processing mode. Computer "flexibility" 
implies general-purpose processing capability. That is, the computer must be 
capable of sophisticated computations, it must be relatively fast, have large 
core and peripheral storage capability and it must be amenable to constant 
algorithm updates and programming changes. Also a multi-program environment 
is essential to allow ready access to a number of researchers simultaneously. 
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Computer requirements for special-purpose processing are best met by a 
large scale general-purpose computer such as the CDC 7600 or Cyber (6000) 
series with ample core and high-speed peripheral storage. This will generally 
assure any given researcher of the flexibility and computer power for his 
particular application. 

The second requirement for special-purpose processing is computer avail- 
ability, and again this will generally require a large scale general-purpose 
computer. The problem here is to ensure all researchers ready interaction 
with the computer. An interactive display system is essential for a facility 
such as NASA JSC, but it is not enough. Only a limited number of researchers 
can use the interactive display at any given time. The remainder must be 
ensured of ready access to the computer as well. Batch processing can ensure 
this provided that turn-around time is kept short. With short turn-around 
results of a given processing run can be analyzed from computer printout and 
at digital display stations. Modifications can then be made and the run re- 
submitted. Several Iterations in this "semi-interactive” mode are possible 
per day. 

In general, processing requirements tend to vary considerably among 
special-purpose runs, since each run is tailored to a particular application. 
Also, since special-purpose processing is research-oriented, algorithms are 
usually in a state of flux and many computer runs will tend to be "debugging" 
runs. Since runs can be profitably time-shared with interactive display 
runs to provide ready access and rapid turn-around. 

Again, because each computer program is tailored to each individual appli- 
cation, program size varies widely for special-purpose runs but is typically on 
the order of 100,000g (about 30 ,OOOj^q) storage locations or less. Thus a large 
scale computer with say 300,000g locations of core storage should be capable of 
providing adequate turn-around in a multi-programming mode. 

An image processing computer facility should have adequate digital storage 
facilities for several frames of ERTS imagery at any given time. This means a 
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reasonable amount of disc storage and medium speed extended core storage (ECS) 
or its equivalent. Also numerous magnetic tape units are required for input/ 
output. 

The current computer installation at NASA JSC meets all these requirements. 

Of coarse this installation is not dedicated to image processing users, but 
access seems to be reasonable. The one modification that we would recommend at 
this stage is a relatively minor one concerning the digital analysis stations 
(DAS). Our experience has been that programming of the DAS is too inflexible. 

More programming effort should be expended on the DAS so that a researcher can 
use the information he has available to best advantage. We are not suggesting 
that the DAS should perform the functions of the general-purpose computer, but 
it should be capable of simple tasks such as contrast enhancemsnt, digital en- 
largement, etc., and at the very least the operator should have the capability 
of changing constants such as those in the header record on MSDS format tapes. 

In other words the current installation overall appears more than adequate 
to meet special purpose processing requirements. The problem arises in trying 
to meet both special-purpose processing and production processing requirements. 

In many cases production processing represents pre-processing to prepare 
data for subsequent special-purpose runs. Thus it must be performed before 
various special-purpose runs. In any case there is a relatively high volume 
requirement so that production processing runs must be performed on a regular 
basis. Moreover production processing tends to involve larger images --usually 
complete frames or at least complete strips of imagery- -requiring correspond- 
ingly larger amounts of storage and processing time than special-purpose 
processing. 

In other words, production processing generally requires large amounts of 
digital storage (both core and peripheral) on a regular basis. On the other 
hand, by the time a computer program reaches the production phase, algorithms are 
more or less well-defined and a minimum of operator interaction is required. Also, 
because of data volume, algorithms tend to employ relatively simple operations. 
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What this means is that a large scale general-purpose computer does not 
provide the most cost-effective answer to the production processing problem* 
Production processing requires the speed and storage capabilities of a large 
scale general-purpose computer, but otherwise tends to under-utilize the 
computer's capabilities* Furthermore in a mixed processing environment pro- 
duction processing runs tend to monopolize the central processor, denying 
access to special-purpose users* 

The problem in designing an image processing facility is to reconcile 
computer requirements for production and special-purpose processing. We at 
Control Data feel that this can best be done through combining the general 
capability of a large scale digital computer with high speed special-purpose 
image processing hardware. Basically the concept is to perform production 
processing in the high speed special purpose hardware under the control of 
the central processor, leaving the central processor free for special purpose 
applications. Thus both sets of requirements can be met simultaneously . 
Production runs can be processed regularly on ahigh-speed, high volume basis, 
while at the same time the general capability of the central processor is 
readily available for special-purpose processing* 

Design of special purpose image processing hardware is very important. 
Through use of parallelism, special-purpose image processing computers can be 
designed for a given application which are one to two orders of magnitude faster 
than the most powerful general purpose computers. For example, a digital change 
detection system has been designed and built for the U.S* Air Force by Control 
Data Corporation [9] that has a processing rate of 250,000 pixels per second. 

The total time required to process a typical 1024 x 2048 frame of imagery on 
this equipment is therefore between 8 and 10 seconds. A simulator for this 
system on the CDC 6600 computer, on the other hand, requires on the order of 
600 seconds of CP time to process the same frame of imagery* Thus the special 
purpose hardware is faster by about a factor of 70. 

The price that must be paid in special-purpose design is flexibility. 
Special purpose computers are just that: computers designed specially for a 
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single application. However, a certain amount of flexibility can be built into 
a special-purpose image processing computer through use of microprogrammable, 
as opposed to hard -wired, components. 

Control Data favors a modular approach in which the image processing com- 
puter is built from a series of micro-programmable mini-computers called Flexible 
Processors . The Flexible Processor (FP) is a specially designed image processing 
computer, which features are described in [14]- A special-purpose computer for 
a given application can be constructed from an array of FP’s with the appropriate 
microprogramming. For a given application, the processing rate can be adjusted 
by changing the number of FP*s. Since the FP’s can operate in parallel, the 
number can be adjusted to meet any desired processing rate. 

Furthermore, an array of FP’s designed for a given application can be 
modified for a different application by simply changing the microprogramming. 

This is considerably simpler than modifying a hard-wired computer, but is, of 
course, more difficult than modifying a FORTRAN program on a general purpose 
computer. 

Obviously, an array of flexible processors may not be the answer to the 
special-purpose programming problem. Algorithms tend to be too sophlsitcated 
and too ill defined, i.e. subject to continual revision. Also various peripherals 
required in such applications are best interfaced with a general purpose computer. 

Such a machine is, however, ideally suited to the production processing 
problem. In production processing applications numerical operations tend to 
be rather simple because of the volume of data to be processed. Further algo- 
rithms tend to be well-defined, i.e. they are more or less fixed. Thus an 
array of processors could be set up and microprogrammed for a given production 
processing problem, then applied to high speed processing on a production basis. 
The number of FP*s could be chosen to yield the desired processing rate at the 
lowest cost. Then more processors can be added as needed to meet higher pro- 
cessing rates or as required for other applications. 
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